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Chapter 1 


Introduction 


This thesis falls within the area of applied mathematics. It raises various problems within 
the area of game theory and offers mathematical solutions to them. 

In this chapter we present the background and the motivation for the research presented 
in this thesis. In Section[L. 1]we first introduce game theory as a theoretical topic of this the- 
sis and as a tool to formalize and solve complex decision making problems. In Section 
we introduce the applications that we consider in this thesis: The optimal toll design prob- 
lem, the problem of energy market liberalization, and the theory of incentives. We conclude 
the chapter with an overview and road map of this thesis, and a survey of the contributions 
to the state of the art in Section 


1.1 Introduction to game theory 


1.1.1 The basics of game theory 
What is game theory? 


Game theory is a branch of applied mathematics used, among others, in the social sciences 
(most notably economics), biology, political science, computer science, and philosophy. 
Game theory attempts to mathematically capture behavior in strategic situations (so-called 
games), in which an individual's success in making choices may depend on the choices of 
others. Game theory was initially developed in order to analyze competitions in which one 
individual does better at another's expense (zero sum games, IS). Later on, game theory 
was expanded in order to treat a much wider class of interactions. 

Traditional applications of game theory attempt to find game equilibria, i.e., sets of 
strategies in which individuals are unlikely to change their behavior. Many equilibrium 
concepts have been developed (e.g., the well-known Nash equilibrium [61]. the Stackel- 
berg equilibrium [92], and the Pareto equilibrium (93h) in an attempt to capture this idea. 
These equilibrium concepts are motivated differently depending on the field of application, 
although they often overlap or coincide. 

Game theory has been widely recognized as an important tool in many fields. Eight 
game theorists have won The Nobel Prize in economics, and John Maynard Smith was 
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awarded the Crafoord prize for his application of game theory to biology. 

The established names of “game theory" (developed from approximately 1930) and 
"theory of differential games" (developed from approximately 1950, parallel to that of op- 
timal control theory) are somewhat unfortunate. “Game theory", especially, appears to be 
directly related to board games; of course it is, but the notion that it is only related to such 
games is far too restrictive. The term "differential game" became a generally accepted name 
for games in which differential equations play an important role. Nowadays the term “dif- 
ferential game" is also being used for other classes of games for which the more general 
term “dynamic games" would be more appropriate. 

The most widely accepted origin of game theory as stated in the literature is found in the 
year 1944, when the book Theory of Games and Economic Behavior by John von Neumann 
and Oskar Morgenstern was published. This theory was developed extensively in the 
1950s by many scholars. Game theory was later explicitly applied to biology in the 1970s. 


Applications of game theory 


The applications of “game theory" and the “theory of differential games" mainly deal with 
economic and political conflict situations, worst-case designs, evolution problems in biol- 
ogy, as well as modeling of war games. However, it is not only the applications in these 
fields that are important; equally important is the development of suitable concepts to de- 
scribe and understand conflict situations. It turns out, for instance, that the role of informa- 
tion - what one player knows compared to others - is very crucial in such problems. 

Scientifically, dynamic game theory can be viewed as the offspring of game theory and 
optimal control theory. Its character, however, is much richer than that of its parents, since 
it involves a dynamic decision process evolving in (discrete or continuous) time, with more 
than one decision maker, each with his/her own cost function and possibly having access to 
different information. 


Conflict as the origin of game theory 


The problems of game theory are often connected with a conflict situation. Although the 
notion of conflict is as old as mankind, the scientific approach dealing with conflict situ- 
ations began relatively recently, around the 1930's, resulting in a still growing stream of 
scientific publications. More and more scientific disciplines devote time and attention to the 
analysis of conflict situations. These disciplines include (applied) mathematics, economics, 
engineering, aeronautics, sociology, politics, and mathematical finance. 

In a conflict situation an individual, also called a player, agent, decision maker, actor, 
or simply person, has to make a decision and each possible decision may lead to a different 
outcome, which is valued differently by that individual. This individual may not be the only 
one deciding in favor of a particular outcome; a series of decisions made by several individ- 
uals may be necessary. If some of the individuals value the possible outcomes differently, 
the seeds of conflict have been sown. 

The individuals involved do not always have complete control over the outcome. Some- 
times there are uncertainties that influence the outcome in an unpredictable way. Under such 
circumstances, the outcome is (partly) based on data not yet known and not determined by 
the other players’ decisions. Sometimes it is said that such data are under the control of 
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"nature" or *God", and that every outcome is caused by the joint or individual actions of 
human beings and “nature” (“God”). 


Basic notions 


So far we have used terms like "decision" and "strategy" without explaining them properly, 
assuming that their meaning is intuitively clear. However, some precision is necessary to 
avoid ambiguities. 

In the following simple example the concepts of decision, action, and strategy (also 
called “decision rule") will be introduced. 

Consider a person who has to decide what to do on a Sunday afternoon, and the options 
are running outdoors or working out in a fitness club. A possible strategy of this individual 
can be framed in these terms: “If the weather is nice, then I will run outside, otherwise I 
will work out.” This is a strategy or a decision rule: what actually will be done depends 
on quantities not yet known and not controlled by the decision maker; the decision maker 
cannot influence the course of the events further, once he/she has fixed his/her strategy. (We 
assume that the decision maker will stick to his/her strategy.) Any consequence of such a 
strategy, after the unknown quantities are realized, is called an action. In a sense, a constant 
strategy (such as an irrevocable decision to go running or come what may) coincides with 
the notion of action. 

In the example above, the alternative actions are to run outdoors and to work out, and 
the actions to be implemented depend on information (the weather), which has to be known 
at the time it is carried out. In general, such information can be of different types. It 
can, for instance, comprise the previous actions of all the other players. As an example, 
consider the following sequence of actions: If he/she is nice to me, I will be nice to him/her. 
The information can also be of a stochastic nature, such as in the running example. Then, 
the actual decision (action) is based on data not yet known and not controlled by other 
players, but instead determined by “nature”. If this “nature” plays no role, the problem is 
deterministic. 


Static versus dynamic game theory 


There is, in fact, no uniformly accepted line of separation between static games, on the one 
hand, and dynamic games, on the other. We shall choose to call a game dynamic if at least 
one player is allowed to use a strategy that depends on previous actions of other players or 
the player herself/himself. If a game is not dynamic, it is static. 


What does “optimality” mean? 


In game theoretic problems, the aim is often to find an optimal strategy for one or more play- 
ers. Optimality, in itself, is not a well defined concept. In non-cooperative games a solution 
in terms of the Nash equilibrium is a specific form of optimality. Such a solution is reached 
if one player cannot improve his/her outcome by altering his/her decision unilaterally. 
Another concept of solution is the one that involves a hierarchy in decision making: one 
or more of the players declare and announce their strategy before the other players choose 
their strategy and the declaring players are in a position to enforce their own strategy upon 
the other players. Such games in which one or more players, called the leaders, declare 
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their strategy first and impose this strategy upon the other players, called the followers, are 
referred to as Stackelberg games. 

If, however, the leaders announce their strategy as a mapping from the followers' deci- 
sion space into their own decision space, we talk about inverse Stackelberg games. Exam- 
ples of inverse Stackelberg games are: 


Think of the leader as the government and of the follower as a citizen. The govern- 
ment fixes how much income tax the citizen has to pay and this tax will depend on 
the income of the citizen. It is up to the citizen to choose how much money he/she 
will earn (by working harder or not). The income tax the government will receive is 
an increasing function of the citizen's earnings, where this tax rule (in many countries 
piecewise linear) was made known ahead of the citizen's decision as to how hard to 
work and, hence, how much to earn. 


The leader is a bank and the follower an investor. The investor can buy stocks, the 
bank acting as an intermediary, with the money he/she has in his/her savings account. 
Suppose he/she buys stocks worth a certain amount of euro. Then the bank will charge 
him/her transaction costs depending on this amount. The transaction costs rule has 
been made known by the bank before the actual transaction takes place. 


The leader is a producer of electricity in a liberalized market and the follower is the 
market (a group of clients) itself. The price of electricity is set as a function of the 
amount of electricity traded (64). 


The leader is a road authority and the followers are drivers in the road network. The 
road authority optimizes system performance by setting tolls on some of the links 
in the network, the drivers make their travel decisions in order to minimize their 
perceived travel time. The travel decisions of the drivers determine their traffic flows 
in the network. If the road authority defines the tolls set in the network as functions 
of the traffic flow in the network, the problem is of the inverse Stackelberg type. 


The leader is a road authority and drivers in a given road network are the followers. 
While the leader sets tolls on some links in the network, the drivers make their travel 
decisions in order to minimize their perceived travel costs. Their travel choices deter- 
mine the traffic flows in the network. If the link tolls are calculated as mappings of 
the traffic flows in the network, this game is of the inverse Stackelberg type. 


1.2 Introduction to the applications studied in this thesis 


In this section the three applications considered in this thesis will be briefly introduced: the 
optimal toll design problem in Section the problem of electricity market liberalization 
in Section|1.2.2j and the theory of incentives in Section|1.2.3 


1.2.1 Optimal toll design 


Let us imagine a road network containing cities and routes connecting them. Individual 
routes have different properties. Some routes, like highways, are wider, have rather high 
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capacities, and have a rather high speed limit, too, whereas other routes, like local ones, are 
narrower, with lower capacities, and have a rather low speed limit. Clearly, the local roads 
will clog easier than the highways. 

Some routes may be tolled. The toll is set by a road authority, which tries to reach its 
own goal, by choosing the routes to be tolled and the amount of money the drivers have to 
pay. 

Within the considered network drivers depart from their origin cities to their destination 
cities, for example, from the city where they live to the city where they work. Each driver 
chooses among the routes available the one that would be optimal for him/her. 

Here the word “optimal” can have a different meaning for different drivers. Some drivers 
need to depart and arrive within a certain time horizon, and do not mind if they have to pay 
toll fees, as long as they do not end up on a congested road (e.g., drivers traveling to work 
everyday, with fixed office hours). For other drivers it is not important when they leave and 
when they arrive, as long as their total travel time is not too high. There are also drivers 
for whom the most important consideration is not to pay any tolls, no matter how slow their 
journey is. Still other drivers pick the most scenic route. 

For each of the drivers a dynamic cost function can be defined. This cost function 
contains a travel-time dependent part, and a part containing the tolls the traveler has to pay 
when traveling from his/her origin to his/her destination. The cost function can also contain 
additional terms, like penalty for deviation from the preferred departure time and penalty for 
deviation from the preferred arrival time. Such a cost function was considered in, e.g., 
Each of the travelers chooses his/her route and his/her departure time so as to minimize 
his/her cost function. The travelers' choices will determine how the traffic spreads over 
the network. In an equilibrium state, no traveler can improve his/her perceived travel costs 
by unilateral change of his/her route or departure time. This coincides with the so-called 
dynamic stochastic user equilibrium [58], or the dynamic deterministic user equilibrium 
id in the complete information case. 

The road authority can set tolls in various manners. In this thesis we compare two 
possible approaches: 


e The road authority sets tolls that can vary in time (dynamic toll), but are not directly 
mapped to the rate of usage of individual routes (traffic-flow invariant toll). The game 
between the road authority setting tolls to reach its goal and travelers attempting to 
minimize their perceived travel costs is defined and solved as a Stackelberg game. 

ande. have been dealing with the optimal toll design problem in this set- 


ting [46 [53 4]. 


e The road authority sets dynamic tolls that are traffic-flow dependent. 'The problem is 
to find optimal toll mappings that would minimize the total travel time of the system 
or maximize the total toll revenue. The game between the road authority setting toll 
mappings to reach its goal and travelers attempting to minimize their travel costs is 
defined and solved as an inverse Stackelberg game. In the situation with second-best 
pricing, i.e., when the toll is not set on all links in the network, the concept of the 


traffic-flow dependent toll is new (See also (za lo [81]. 


Although we formulate the optimal toll design problem problem in a general manner, such 
that a solution of the problem exists for wide class of objective functions and user equilibria 
models 


6 1 Introduction 


The tolls maximizing the total toll revenue of the system will be much higher than those 
minimizing the total travel time of the system, as one would intuitively expect. 

Both problems mentioned are N Mt which is why we use advanced heuristic 
methods, like a neural networks approach [80], to find a satisfying solution. 

In some of our case studies the optimal traffic-flow dependent toll is a decreasing func- 
tion of link traffic flow (rather than increasing as one would assume). This phenomenon is 
further discussed in Chapters [4]and[5] 


1.2.2 Electricity market liberalization 


The European electricity market is currently in the midst of drastic transformation from 
monopolistic, national, and state-owned electricity producers (firms) to a market with com- 
peting, private, and often multinational firms. The aim of liberalization is to decrease the 
sales price of electricity and to bring about more cost efficient electricity production. Little 
is known about other effects of liberalization, like impacts of this process on environment. 

To get more insight into the impacts of liberalization, we introduce a game-theoretic 
model with electricity producers in various countries as players (see also HEN Various 
scenarios of a firms’ behavior, depending on the properties and market power of the firms, 
but also on the strategies of European policy makers, are included in the model. The model 
encompasses eight European countries: Belgium, Denmark, Finland, France, Germany, The 
Netherlands, Norway, Sweden. 

The firms in individual countries generate electricity by means of different technolo- 
gies. A producer can own one or more power plants of different types,! for which the total 
capacity for each technology as well as the variable production costs are given. Produc- 
ers maximize their pay-offs by choosing the amount of electricity to produce with various 
technologies for various load modes. Firm pay-offs consist in the income from sales of 
electricity in regional markets minus the (variable) costs of production. 

There are limitations on transportation possibilities of electricity, and production ca- 
pacity of electricity is fixed in the short term. The electricity demand for each country is 
exogenous. Electricity trade is only feasible with neighboring countries. Emissions are 
assigned to producers based on the actual technology used and can also be limited. 

Real data used for computations are consumers’ demands for electricity per region, sup- 
ply data (generation capacity and cost), trade data (interconnection capacity), data pertain- 
ing to distribution losses, and emission factors. 

The outcomes of our case studies show that liberalization decreases electricity prices 
and may decrease production of emissions, provided that restrictions on the electricity pro- 
duction are set well. 

The problem of electricity market liberalization is dealt with in Chapter [6] 


1.2.3 Theory of incentives 


Another application of game theory is so-called theory of incentives. This theory deals with 
so-called principal-agent models (51152 [sd (d which are an example of the one-leader- 
one-follower inverse Stackelberg game introduced in Chapter[3] 


‘Bach power plant corresponds to one specific technology, but more power plants can be owned by one pro- 
ducer. 
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Consider a bilateral relationship, in which a principal contracts an agent to be respon- 
sible for the production of some good. The principal has to pay the agent for the good. 
The salary which the principal offers to the agent for the production of a certain number of 
products will be decided by the principal. The principal draws up a contract in which he 
specifies the quantity of goods he wants and the salary that he is going to pay to the agent 
for obtaining the demanded products. 

Conflicting objectives and decentralized information are two basic ingredients of incen- 
tive theory. The essential paradigm of the analysis of market behavior by economists is one 
in which economic agents pursue, at least to some extent, their private interests. 

The agent can have private information. This private information can be of two types: 
either the agent can take an action unobserved by the principal (the case of moral hazard), 
or the agent has some private knowledge about his/her cost or valuation that is ignored by 
the principal (the case of adverse selection). In the incentive theory the main problem is to 
find an optimal strategy for the principal, when he does not have a complete information 
about the agent. 

We will introduce several incentive problems and discuss optimal strategies for the prin- 
cipal with different scenarios of the agent’s behavior (See also [78]. These problems fall 
within the inverse Stackelberg games. 

The theory of incentives is dealt with in Chapter[7] 


1.3 Overview of this thesis 


1.3.1 Thesis outline 
This thesis is organized as follows: 


In Chapter [2]the foundations of classical game theory are introduced. 


In Chapter Blwe introduce the extension of classical game theory that we use in this 
thesis, the so-called inverse Stackelberg games. 


In Chapter [4] we propose an extension of the static optimal toll design problem to 
a situation with a traffic flow-dependent toll. We develop a neural networks-based 
algorithm to solve this problem. 


In Chapter[5]we propose an extension of the dynamic optimal toll design problem to 
a situation with a traffic-flow dependent toll. Also here we propose a neural networks- 
based algorithm to solve this problem. 


In Chapter[6]we define the problem of a liberalized European electricity market. We 
present various scenarios differing in the electricity producers' behavior and solve 
these problems analytically or using a numerical algorithm implemented in Matlab. 


In Chapter[7]we introduce the theory of incentives as a subset of the inverse Stackel- 
berg problems, and we present and solve different principal-agent problems. 


Chapter [8] summarizes the results of this thesis and outlines directions for future 
research. 
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Figure 1.1: Road map. Arrows indicate recommended reading direction 


1.3.2 Road map 


Figure[I.1]illustrates a grouping of the chapters in related subjects and an ordering in which 
the chapters can be read. It is suggested to read the chapters in the order as they appear in this 
thesis. Chapter[I]contains a general introduction to the topic in this thesis, and is therefore 
suggested to be red first. Chapters[2]andB]both focus on game theory. Chapter[2]focuses on 
"classical" game theory and explains its main concepts used in this thesis. Chapter[3]deals 
with so-called inverse Stackelberg games, and is one of the contributions of this thesis. 
Chapters [6] and [7] deal with game theory applications studied in this thesis. It is 
therefore suggested to read Chapters [2]and[3]before Chapters [4] [5] [6] and[7] Both Chapters 
[4] and [5] focus on bilevel optimal toll design problem, the former on its static version, the 
latter on its dynamic version. It is suggested to read Chapter M]before Chapter[5] Chapter [8] 
summarizes the results of this thesis and gives directions for future research. This chapter 
should be read as the last chapter. 


Main contributions 


The main contributions of the research described in this PhD thesis with respect to game 
theory are the following: 


e The concept of an inverse Stackelberg game as generalization of a Stackelberg game 
is introduced and studied, mainly by means of examples. So far, almost no literature 
dealing with inverse Stackelberg games exists, thought the concept has been known 
for some time. 


1.3 Overview of this thesis 9 


e Possible ways of how to find a (sub-)optimal solution of an inverse Stackelberg game 
are proposed. 


e Ina general game theoretical framework, it is shown that under the same initial con- 
ditions an inverse Stackelberg game can never bring a worse outcome than a related 
Stackelberg game, as the Stackelberg strategy is a special case of the Stackelberg 
strategy. 


The main contributions of the research described in this PhD thesis with respect to the game 
theory application in the optimal toll design problem are the following: 


e A concept of a traffic-flow dependent toll in the optimal toll design problem is defined 
for both the static and the dynamic optimal toll design problem. 


e Properties of the optimal toll design problem are discussed. 


e A neural-networks based algorithm for solving the optimal toll design problem with 
a traffic-flow dependent toll is proposed. 


e Itis shown that the road authority can never be worse-off with a traffic-flow dependent 
toll than with a traffic-flow invariant toll, since the traffic-flow invariant toll is a trivial 
case of the traffic-flow dependent toll. 


The main contributions of the research described in this PhD thesis with respect to the 
game theory applications in the energy market liberalization problem are the following: 


e A model of a liberalized electricity market, involving 8 European countries, is pro- 
posed. 


e Different game theory concepts are applied to this model and it is shown that a mo- 
nopolistic or a duopolistic market yields higher electricity prices than a highly com- 
petitive market. 


The main contributions of the research described in this PhD thesis with respect to the 
game theory application in theory of incentives are the following: 


e A classical principal-agent model is an inverse Stackelberg game. 


e Examples of this game are given and solved analytically. 


Chapter 2 


Results from Classical Game 
Theory 


In this chapter some classical results from game theory, used in this thesis, will be recapitu- 
lated. 


2.1 Preliminaries 


Definition 2.1 (Game) 
A game is the interaction among rational, mutually aware players, where the decisions of 
some players impacts the payoffs of others. A game is described by its players, each player's 
strategies, and the resulting costs for each outcome. Additionally, in sequential games, the 
game stipulates the timing (or order) of moves. 


Note that a player's strategy in a game is a complete plan of decision (action) for what- 
ever situation might arise; this fully determines the player's behavior. A player's strategy 
will determine the decision (action) the player will take at any stage of the game, for every 
possible history of play up to that stage. A strategy profile is a set of strategies for each 
player which fully specifies all actions in a game. A strategy profile must include one and 
only one strategy for every player. A pure strategy defines a specific move or action that a 
player will follow in every possible attainable situation in a game. Such moves may not be 
random, or drawn from a distribution, as in the case of mixed strategies. A mixed strategy is 
a strategy consisting of possible moves and a probability distribution (collection of weights) 
which corresponds to how frequently each move is to be played. One can regard a pure 
strategy as a degenerate case of a mixed strategy, in which that particular pure strategy is 
selected with probability 1 and every other strategy with probability 0. 

We will use the following notation: Let D; be a decision space (set of possible decisions) 
for the i-th player in an n-person noncooperative game. Let u; € D; be a decision of the i-th 


def i: def 
player. Let D £ Di X Dy X -+ X Dy be the set of decision spaces. Vector u £ (u1,...,Un) 


will be called a decision profile, vector u_; gef (ui, ..., Uil, Uit, ---, Un) Will be the 
decision profile without the i-th decision. The objective function for the i-th player will 
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be denoted by 7j, where 7; = 7;(u). If player i € (1,...,n) chooses decision u; resulting 
in decision profile u = (u1,..., Un), player i obtains outcome 9;(u). Note that the objective 
function is individual and depends on the decision profile chosen, i.e., on the decision taken 
by player i as well as on the decisions of all the other players. 

Below we will use the notion u; not only for the decision of the i-th player, but also for 
the value of such a decision. In Chapters [6] and [7] different notions for the players' 
decisions will be introduced to avoid misunderstanding. 


2.2 Nash equilibrium 


A Nash equilibrium is a set of strategies for finite, non-cooperative games between two 
or more players whereby no player can improve his/her payoff by changing their strategy. 
Each player's strategy is an “optimal” response based on the anticipated rational strategy of 
the other player(s) in the game. 


Definition 2.2 (Nash equilibrium) 

A decision profile u* = (uj,...,u;,) € Dis in a Nash equilibrium (NE) if no unilateral devia- 
tion in decision by any single player is profitable for that player, i.e., vie (1,...,n], we 
Di, Uj Æ u; 


* * x ok * * * * * 
Jig eus HE Hid e) S m oH oa WEM) 


A game can have a pure strategy Nash equilibrium or an Nash equilibrium in its mixed 
extension. Nash proved that, if we allow mixed strategies (players choose strategies ran- 
domly according to pre-assigned probabilities), then every n-player game in which every 
player can choose from finitely many actions admits at least one Nash equilibrium. 

Players are in a Nash equilibrium if each one is making the best decision that he/she 
can, taking into account the decisions of the others. However, the Nash equilibrium does not 
necessarily mean the best cumulative payoff for all the players involved; in many cases all 
the players might improve their payoffs if they could somehow agree on strategies different 
from the Nash equilibrium. 


Remark 2.1 In Chapter[4]the so-called Wardrop equilibrium will be introduced, as a lim- 
iting case of the Nash equilibrium applied in macroscopic traffic modeling. The Wardrop 
equilibrium is the Nash equilibrium with a very large number of players. Then the contri- 
bution of a single player to the outcome of the game tends to zero. 


2.3 Stackelberg equilibria and terminology 


For the sake of simplicity we will consider a game with two players only. 
Let us consider two players, called Leader and Follower, respectively, each having 
his/her own cost function, 


I (uL,ur), Ip (uL,ur), 
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where uj,ur € IR. Each player wants to choose his/her own decision variable in such a 
way as to minimize his/her own cost function. In the Stackelberg equilibrium concept one 
player, the leader, announces his/her decision up, which is subsequently made known to the 
other player, the follower. With this knowledge, the follower chooses his/her up. Hence up 
becomes a function of uj, written as 


UF = IE (uL) ; 
which is determined through the relation 
nu Jr (UL, ug) = Ir (ur; Ig(ur)). 


Here it is assumed that this minimum exists and that it is unique for each possible choice 
uy of the leader. The function /p(-) is sometimes called a reaction function (i.e. it indicates 
how the follower will react upon the leader's decision). Before the leader announces his/her 
decision ur, he/she will realize how the follower will react and hence the leader will choose, 
and subsequently announce, uj, such as to minimize J, (ur, Ip(ur)). 


Example 2.1 
Suppose 
JL(up.ug) = (up— 5)? +u?, — jp(up, ug) = uz + ug — uu. 
The reaction curve lp is given by ur = ju (it is easily found by differentiating 7j, with 
respect to ur)and hence uj, will be chosen such as to minimize 


1 2 
(zu — s) + ul. 


which immediately results in uj, — 2. With this decision of the leader the follower will 
choose up = 1. The costs for the leader and follower are given by 20 and 3, respectively. 


Note that the best that the leader can obtain is 
min JL (ur, up) 
up EDL,UFEDF 


We will refer to this value as to the team minimum. 


2.4 Open loop versus closed loop 


These concepts appear in games in which time evolution plays a role. 

In open-loop information patterns a strategy only depends on the initial state, at the 
beginning of a game. In closed-loop information patterns the strategy depends on the current 
state, i.e., the state at the moment that a decision has to be made. 


2.5 Tools for one-person optimization 


In this section we will introduce some optimization techniques adopted from control sys- 
tem theory and used in this thesis. For more details about individual techniques, see 
In Section [2.5.1] the dynamic programming approach for continuous-time systems will be 
introduced. In Section[2.5.2]the minimum principle will be introduced. Section[2.5.3]deals 
with affine quadratic continuous-time optimal control problems. 
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2.5.1 Dynamic programming for continuous-time systems 


The method of dynamic programming is based on the principle of optimality which states 
that an optimal strategy has the property that, whatever the initial state and time are, all 
remaining decisions (from that particular initial state and particular initial time onwards) 
must also constitute an optimal strategy. To exploit this principle, one has to work back- 
wards in time, starting at all possible final states with the corresponding final times. The 
dynamic programming approach, when applied to the single criterion optimization problems 
defined in continuous time, leads to a partial differential equation, known as the Hamilton- 
Jacobi-Bellman (HJB) equation. We will consider the problem defined as finding decision 
u minimizing cost L(u), where 


x0) =F(tx(0),u)), =, 120, Q.D 
u(t) — Y(t,x(t)) €S, YET, (2.2) 
L(u) = f atout) dt 4-q(T,x(T)), (2.3) 

T = min(t 1 (r,x()) —0). (2.4) 


Here f indicates the time, x denotes dx/dt. The state x of this model evolves in time ac- 
cording to the differential equation x = f (t,x(t),u(t)). In general the state x can be an 
n-dimensional vector (written as x € R”) and t € [0,7], where T > 0 represents the fixed 
final time. Under suitable conditions on the function f, the time evolution of x is uniquely 
determined by the differential equation. A scalar function / defines an n-dimensional smooth 
manifold in the product space IR" x R+, and the class of all admissible closed-loop strategies 
I'. The so-called value function 


V(t,x) = min | gts.) ds +q (T‚x(T)) |, (2.5) 


satisfying the boundary condition 
V(T,x) 2 q(T,x) along U(T‚x)=0. (2.6) 


describes the minimal cost-to-go from any initial state x and any initial time t. If V is con- 
tinuously differentiable, the principle of optimality yields the following HJB equation: 
dV (t,x) OV (t,x) 


ot mm —3. (ou) + (F454) ’ (2.7) 


which takes as the boundary condition. 


Theorem 2.2 (Sufficiency) If a continuously differentiable function V (t,x) can be found 
that satisfies the HJB equation subject to the boundary condition (2.6), then it gener- 
ates the optimal strategy through the static (pointwise) minimization problem defined by the 
right-hand side of (2.7). 


Proof: See ish. 
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2.5.2 The minimum principle 
Let V from be twice continuously differentiable. Let function H be defined as 


B xu) SKED p) + (rna) 


Then equation (2.7) for u* minimizing Éf(t,x,u) can be written as 


ƏV (t, x) 
ot 


Since V is twice continuously differentiable, differentiation of with respect to x and t 
yields 


+H (t,x,u*) — 0. (2.8) 


d de de)” ma Oe Oe (om 


og d (5) oVof oH du* 
It can be seen that oH = 0 for u = u* according to (2.8), if u is not constrained (If there 
are constraints on u, and u* happens to be on the boundary, then it can be shown that 
oH Q^ — 0.). In all cases, equation becomes 


dg d/AV\ Wof | 
ES a) xz) (2.10) 


Let x* denote the state trajectory corresponding to u*. By introducing the so-called costate 
def ; 
vector, Mt) = VD), (2.10) can be rewritten as 


dp’ d kk ko ð * pk 
d = ET [g(t,x „u ) - A(t)f(t.x „u )] = —5 H(t. pox „u ), (2.11) 
where H(t, p,x,u) EL e(t x,u) +p'f(t,x,u). Since I(T,x) = 0 for the final time T, T can 
be regarded as a function of the state, i.e., T = T (x). The boundary condition for p(t) is 
determined by 
OV (T(x*),x*)  9q(T(x"),x*) 
T) = = 2L. 2.12 

p (T) àx x Q.12) 
Under the assumption that the value function V (t,x) is twice continuously differentiable, the 
optimal control u*(t) and corresponding trajectory x* (f) satisfies the following canonical 
equation: 


oH \' 
x(t) = (5x) = f(t,x*,u*), x(to) = xo, (2.13) 
X(t) = s Maes ie) (2.14) 
ox 
N(T)= ars kene det (2.15) 
X 
H(t, A,x,u) E e(t, x,u) -- A f (t,x,u), (2.16) 


u* (t) = argmin H(t, A,x*,u). (2.17) 
uc 
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In the derivation of (2.13)-(2.17) the controls have been assumed to be functions of time 
and state. If the control functions are dependent on time only, one obtains the following 
(Pontryagin) principle. 


Theorem 2.3 (Pontryagin minimum principle) Consider the optimal control problem de- 
fined by 2.1)-@.13) and under the open-loop information structure. If the functions f, g, q, 
and l are continuously differentiable in x and continuous in t and u, then relations (2.13)— 
provide a set of necessary conditions for the optimal control and the corresponding 
optimal trajectory to satisfy. 


Proof: See (47). 


2.5.3 Affine quadratic optimal control problems 


Let a system be given by 
X — A(t)x(t) 4- B(t)u(t)+c(t), x(0) = xo. (2.18) 


Let the function to be minimized be defined as 


T 

L(u) = Sx) Qyx(T) + I n (X Qx + 2x p +u'Ru) dr, (2.19) 
where x(t) € R”, u(t) € R", 0 € t € T and T is fixed. A(-), B(-), Q(-) > 0, R(-) > 0 are 
matrices of appropriate dimensions and with continuous entries on [0,7]. The matrix Of is 
nonnegative-definite, and c(-) and p(-) are continuous vector-valued functions, taking values 
in IR". Furthermore, we adopt the feedback information pattern and take a typical control 
strategy as a continuous mapping y: [0,7] x IR" — IR". The space of all such strategies will 
be denoted by I’. The optimal control problem is to find a y* € I such that 


I) <I), WEL, (2.20) 


where 


IL), with u(-) 2 yx). (2.21) 


Since 9 (y) is quadratic in xo (see [ip and the minimum cost-to-go, starting from an arbitrary 
t € [0, T] at an arbitrary point x € IR", is quadratic in x, we can prove that there exists a 
continuously differentiable value function of the form 


1 
V(t,x) = z S(t)x + E (re mr) (2.22) 
that satisfies (2.7). Here S is a symmetric n x n matrix with continuously differentiable en- 
tries, k(-) is a continuously differentiable n-vector, and m(-) is a continuously differentiable 


function. 


Substitution of (2.22) into leads to 


Í y ; 1 1 
ZS xk n= min (Sx -- K)' (Ax + Bu 4- c) + 5 Qut xp + 5" Ru ; (2.23) 


2.5 Tools for one-person optimization 


Minimization of the right hand side leads to 


u* (t) = Y (t,x(t)) = —R7'B’ [S(t)x(t) + k(t)]. 


By substituting (2.24) into (2.23) we obtain the following conditions: 


$--SA--A'S- SBR !B'S--Q—0, S(T)—Qy, 
k+(A—BR IBS) k--Sc--p—0, k(T)=0, 


1 
m+kc- 5K BR 'B'k =0, m(T)=0. 
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(2.24) 


(2.25) 
(2.26) 


(2.27) 


Proposition 2.5.1 The affine quadratic continuous-time optimal control problems (2.18)— 
(2.19) admits a unique optimum feedback controller y* which is given by (2.24), where S(-), 
k(-), and m(-) uniquely satisfy (2.25)-(2.27). The minimum value of the cost function is 


I) = 5965(0) x0 +K (0) xo + m(0). 


Proof: See [5]. 


Chapter 3 


Inverse Stackelberg Games 


Parts of the research presented in this chapter have been presented in 78]. In SectionB.T]we 
deal with static inverse Stackelberg games. Dynamic inverse Stackelberg problems are in- 
troduced in Section[3.2] Conclusions, possible extensions, and future research are discussed 
in Section 


3.1 Static inverse Stackelberg games and equilibria 


In Section the static inverse Stackelberg game with one leader and one follower is 
introduced, in Section|3.1.2|static inverse Stackelberg problems with one leader and multiple 
followers will be dealt with. 


3.1.1 One leader - one follower games 


Let us consider a game with one leader and one follower, each having his/her cost function 
JL (UL, UF), JF (UL, ur), 


to be minimized. In the inverse Stackelberg game the leader does not announce the scalar 
ur, as in the Stackelberg game introduced in Section[2.3] but a function y; (-), which maps 
ur into uL. 

Given the function Yyı(-), the follower's optimal choice of ur, indicated by an asterisk, 
satisfies 


Up = arg min Jr (Yr (ug) ug). (3.1) 


The leader, before announcing his/her y; (-), will realize how the follower will play, and 
he/she should exploit this knowledge in order to choose the best possible yı-function, such 
that ultimately his/her own cost becomes as small as possible. Symbolically this can be 
written as 


WC) = arg min (fur) ur (1. ())). (3.2) 
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The problem belongs to the field of composed functions [50], and is therefore in gen- 
eral very difficult to solve. In general it is very complicated to find an analytical solution of 
(3.2), if it exists at all. 

However, if the leader knows what he/she can achieve (in terms of minimal costs) and 
what has to be done by all players to reach this outcome, the leader may be able to persuade 
other players to help him/her to reach this goal (i.e, the value of the leader's cost function 
obtained if all players minimize it), as shown in Example [3.1] If it is unknown what the 
leader can achieve in terms of minimal costs, finding the leader's optimal y-strategy is 
generally very difficult. 


Example 3.1 
Suppose the cost functions are those of Example i.e., 


IL(uL,up) = (up— 5) Hui, Ip(uL,up) =u? + ug — uur. 
If both the leader and the follower would minimize J, (ur, ug), the follower totally disre- 
garding his/her own cost function, the leader would obtain the team minimum 


min JL (ur, ug) = IL(0,5) — 0. 
up € Dr .upgc DE 


To obtain the team minimum in the inverse Stackelberg game the leader should choose the 
Yr-curve in such a way that the point (ur, ug) = (0,5) lies on this curve and, moreover, that 
the set 

{fr (ur), ur) ur € Dr} 


does not have other points in common with the set 
Tur. ug) [Jr (uz, ur) < Jr (0,5)}- 


An example of such a curve is un = 2ur — 10. Clearly, this is the only linear curve satisfying 
the requirements. 
With this choice of the leader, the best for the follower to do is to minimize 


Jr (Zur — 10, ug), 


which leads to ur = 5. Then uy, = 0 and the leader obtains his/her team minimum in spite 
of the fact that the follower minimized his/her own cost function (although subject to the 
constraint un = yr (ur) = 2ur — 10). 


The following two examples show situations in which the team minimum cannot be reached. 


Example 3.2 
Let DL = (o, B}, Dr = {7,5}, a, B, y,6 € R. If the optimal strategies for the leader and for 
the follower are !: 


: Ks if up —Yy, =} if u —, 
m ö, if up =Q. 


Clearly, in this situation the leader cannot reach the best-possible (team minimum) outcome 
in the deterministic sense. However, a mixed strategy solution can be found. 


'It is easy to define cost functions, corresponding to these optimal strategies. 
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Example 3.3 


Let us consider Example with restriction of the decision spaces for both the leader and 


the follower, DL © [-4,3], Dr E [—5,7]. The worst that can happen to the follower is 


characterized by min, max,, Jr, which is realized for ur = —2, ur = —4 (Ir(—4,—2) = 
12). In optimal case the leader obtains 


min Jr, subject to Jp € 7p(—4,—2) = 12. 
UL up 


The solution uy, and ur will be indicated by u; ul. An £-optimal choice for the leader is 


—4 for —~5<up<ub—e 

up = YL (ug) E + , i F , 

uy, for up—€ <up <7, 

where € is an arbitrarily small nonnegative number. If € > 0, the solution is unique, if £ = 0 
the follower can respond in a non-unique way. 


Example will deal with a situation, in which the leader does not know in advance, what 
he/she can achieve. 


Example 3.4 

Let us consider an inverse Stackelberg game, in which the follower minimizes the sum of 
f (up) and yL (ug), where yL : Dp > RO, Yı(0) = 0, is chosen by the leader and f : Dp > R 
is a given function, i.e., 


up — arg min (f (ug) 4- vi (ur)), (3.3) 


upgc DE 


while the leader maximizes Yr (ur), i.e., 
LC) = arg max (ur). (3.4) 
yv. C) 


This example can be interpreted as follows: The leader is a bank and the follower is 
an investor. The investor maximizes his wealth — f (ur) — Yı (ur), where ur [euro] is the 
investment. For ur = 0 no transaction takes place. Let y; (-) represent transaction costs 
function, i.e., if the investor makes investment decision up, he has to pay transaction costs 
of yL (ur) [euro]. Since the investor should be secured of a maximum cost f(0) by playing 
up = 0, we assume that he/she will only take up-values from the set U defined as 

def 
U = {ur : f (ur) < f(0)). 

In practice the function f would depend on the market situation. Let us consider f 

defined as follows 


f (ur) € (ug — 0)? +B, 


with & > 0. Then f(0) = o? + B and, therefore, U = [0,20]. 
We will try to find an £-optimal y_-function in an ad-hoc way. Since y, (ur) is included 
in the follower's cost function, we will try to check how different choices of y, influence the 
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outcome of the game. Intuitively, it seems to be reasonable to choose yj -function defined as 
follows: 


nonnegative, elsewhere, er 


nE (F(0)—f(ur))(1-e), if 0<up<2a, 


where e | 0. 

With the y; -function defined by the optimal follower’s decision is up = a, the 
follower's costs are o + B — ae and the leader's profit is (1— eo’. The leader (bank) reaps 
essentially all the follower's (investor's) profits (the latter would have been min,» f (ur) = 
a if the transaction costs would have been identically zero). Note that the e-optimal Yy,- 
function of the leader is non-unique; another choice, similarly advantageous to the bank, 
would be 

o?—e, if ur #0; 
Your) = { 0, if ur=0, 


where £ | 0. Then the outcomes for the leader and the follower are o — € and o? + B —e, 
respectively. Note that for £ | 0 the outcomes of the two games do not differ. 


Note that an upper bound for the leader's profit in ExampleB.4]is 
Jr (ur = ug) — Ir(ur = 0), 


where u% is the optimal decision of the follower in absence of transaction costs. 


3.1.2. One leader - multiple followers games 


If there are two or more followers in the decision problem, the relationship, which deter- 
mines the solution concept to be adopted between the followers, must be specified. Let 7 
be the set of all followers and let |F| denotes the number of elements in the set 7 . Let DL, 
Dr, be decision spaces of the leader and the i-th follower, respectively, i = 1,...,|F]. 

An inverse Stackelberg strategy for the leader is a mapping YL : Dr, X X Dp, DL. 
This mapping can also be a vector-valued function, if DL € R”, n € N. Suppose that uf = 


(ui, ee ub.) is the |F|-tuple of the follower's decisions desired by the leader. We say that 
Yı is a dominant strategy solution for the leader, if 
arg min JF; (v. (ur, TE Lr» „ÜR ee ugs.) = UF,» (3.6) 
up; CDF; 


with arbitrary up,, Vj #i,i=2,...,M. 
If the followers minimize their own cost functions, being noncooperative among them- 
selves, a natural solution concept for their behavior is the Nash equilibrium, introduced in 


Section 


Example 3.5 [Followers minimizing their own costs] 
Consider three players, the leader £ and two followers 71, 75, with decision variables ur, 
up,, Up,, respectively. The decision spaces for the followers are the real numbers, i.e., 
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Dr, = Dr, = R, while Dy = R2. The followers’ cost functions (to be minimized) are given 
as follows: 


2 
1 1 1 
Jp, = (ul -2) + ue + ug, — ul ur, = up, 


2 
Jp, = C t 1) + up,” +up, 2u ur, — 2u” UF, 


and they are playing a Nash game among themselves. The leader has the cost function (to 
be minimized) 


n= (ul) " 1) + (u + 1) + (ug; — 1)? + (up, +1} (3.7) 
Itis obvious that the team minimum for the leader is in the point 
(u u? ur, ‚ur, =(1,-1,1,-1). 
An optimal strategy for the leader is to choose linear functions 
im TEL ul) =E(up, +1)-1, EER. (3.8) 
Because the parameter € can vary, the strategy is nonunique. This strategy yields the 


team minimum for the leader. Outcomes for the leader, the first follower, and the second 
follower are then 0, 3, and 2, respectively. 


In some cases the leader can decouple the followers from each other, and, therefore, the 
leader can control each of the followers’ cost functions separately. 


Example 3.6 [Example of a decoupling strategy of the leader] 


Let us consider three players, one leader £ with uj, = (usu?) (DL = (R;)*) and two 


followers F1, F2 with decision variables ur, and up,, respectively, and decision spaces 
Dr, = Dp, = R+. All players want to minimize their cost functions defined as 


IL = (ur, -- ug). , 


1 
Jp, = (up, — 1) + ur, n 


2 
Jp, — (up, — 1)? + 2ur, +u®. 


If the leader applies ul) = Pur) = Zur, — 2up,, it will induce ul) = 0 regardless 


of the value of up, , and similarly uc = if?) (up, ) = Zur, will induce u — 0 for all values 


Ur), and hence 
— (X0 (2) 
YL (ur, ;up;) = (ug, ) Yi (ur, ) 
constitutes a dominant strategy. The leader can control cost functions of each follower 


separately. However, such a solution does not exist generally, because the cost functions of 
the followers may not have the required structure. 
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Example 3.7 [Followers in a Wardrop equilibrium] 

Consider again three players, the leader £ and the followers 71, 75, where the decision 
spaces are defined as D, = (ur, ur, — aur, - b, a,b € R}, Dr, = Dp, = RD. Additionally, 
let the leader's objective function be 


JL = (ur, y | (ur) up, UF, — Ì3 UF}; (3.9) 
and let the followers have cost functions 
Jp, = up, t uL, Jp, = 10ur,, (3.10) 
respectively. Additionally, let the following constraints have to be satisfied: 
up, T up, = 10, Jp, = Jp. (3.11) 
The leader can in advance compute that the optimal followers’ reactions to his uj, = aur, +b 
are 


b-100  _ 10a+b+10 
at+ll’ P" a+ll 


up, — — (3.12) 
for any choice of a and b. Hence, the leader minimizes J, with up, and ur, given by (3.12), 
Le, 
_ 1004? — 1100a 4-33ab + 3 b* — 237 b + 5800 

(11 4- a? 


Minimization of this function with respect to a and b leads to a = —2/115* 4- 79/11 where 
b is free. This choice of a yields uy, = 79/2 and JL = 37/4, while the optimal up, and ur, 
are 11/2 and 9/2 and yield 7p, — 7p, — 45. 


JL 


Remark 3.1 The interpretation of in Example[3.7]is as follows: 10 is the number 
of drivers traveling from origin o to destination d choosing among two links l1, l2, ur, 
and ur, are traffic flows on link /; and link I», respectively. The travelers’ choices of links 
determine the traffic flows in the network. Link /; is tolled with traffic-flow dependent toll 
uL E aur, +b (a,b € R+), Ir, (ur, ur.) is the cost of using link /ı and Jp, (ur, ) is the cost 
on the link l2. Equations can be interpreted as Wardrop equilibrium among travelers 
194], provided that both links are used. In Chapters [4]and[5] more problems of this type will 
be considered. While in reality the traffic flows are integer-valued, in our case studies we 
consider real traffic flows. 


Example 3.8 
Consider the game with two followers, with 


2 2 
Jr, = ug, — ur, UL + 2uj, 


2 2 
Jp, = ug, — 2up, ur + Sup, 
and one leader with the cost function 


Vu ur + Zur, up + Sup, u + ug, + up, Ld. 
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The team minimum of J would be achieved if 
up, = —8/25, up, — —20/25, and uy =8/25. 
The leader will try to obtain his team minimum by right choice of the coefficients a, B, din 
up = YL(UF, ‚ur, ) = Our, + Bur, +ô. 


If he is successful with linear functions, there is no necessity to consider the larger class 
of nonlinear functions. We derive three (linear) equations for the coefficients a, D, 5. The 


first one is obtained by the fact that the absolute minimum must lie on the curve ur = 
x : 07g. (ur, ; g 
Our, + Bur, +03. The second and third ones are obtained by Oe n qum e. =0,i=1,2. 
i 


The equations are 
—8a—208 +255 = 8, 
16a—20B—255 = 16, 
—804-80p—508 = 40, 
which results in 


7 8 332 
15° 375 


3 
a s B (3.13) 
The YyL-function with coefficients given by leads to the leader's team minimum. 
In other words, he/she cannot do better. 
A different approach to find the solution of the problem could be described as fol- 
lows: Consider the constant level curve Jp, (ur, ‚ur) through this point, i.e. Jp, (ur, ur) = 
928/625. This curve determines ur, as a function of up,. By taking the total derivative 


of Ir, (up, uL) = 928/625 with respect to ur, one obtains A = 2 for (ur, up, ur) = 
(—8/25, —20/25,8/25). By considering the constant level curve Jp, (ur, ur.) through the 


same point, one obtains similarly ik Hence, if a linear y, function exists, it must be 
dur, 15 


of the form 


UL = YL (ur, ,UF, ) = Qup, + Bur, +6 


with & = 3, B= 5: Now 8 is obtained by the fact that the curve ur, = '/j, must pass through 
the point (ur, ‚ur, ur.) = (—8/25,—20/25,8/25.) This yields 6 = >>. 


3.2 Dynamic inverse Stackelberg games and equilibria 


The dynamic inverse Stacklelberg game with one leader and one follower is introduced Sec- 
tion[3.2.1] In Section [3.2.2]the inverse Stackelberg problems with one leader and multiple 
followers will be dealt with. Note that we focus on continuous-time dynamics (as oppose to 
dynamic problems considered in Chapter[5) although the discretized versions of the prob- 
lems are used for their solution. 
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3.2.1 One leader - one follower games 


Let us consider a dynamic system defined by 
X-—f(xup,ug),  x(0)- xo. 


Here f indicates the time, x denotes dx/dt. The state x of this model evolves in time accord- 
ing to the differential equation x — f. 

In general the state x can be an n-dimensional vector (written as x € R"), but we restrict 
ourselves ton = 1 and t € [0, T], where T > 0 represents the fixed final time. The quantities 
uj, i = L,F, are scalar functions of time possibly restricted to a certain set u;(t) € U;(t), 
which will be specified later. The function u; must be chosen in such a way as to minimize 
the cost function 


| ee.) ar) 


both g; and g; are scalar functions and are assumed to satisfy certain regularity conditions, to 
be introduced later, such that the cost functions are well defined. Under suitable conditions 
on the function f(x, ur, ur), the time evolution of x is uniquely determined by the differential 
equation. There are no restrictions on x(T ), it is the so-called free endpoint problem. The 
specific problem on which we will concentrate now is: 


X= f(x,ug), x(0) — xo, 


mine — min (q(x(7))-+ [eua f leer). 


T 
max Jp = max | yi (ur(t)) dt. 
Y. C) LC) JO 


The function Yy, is up to the choice of the leader and satisfies 


y.(0) 20, v.) 2 0, Yr (ur) = vL (ur). 


Similarly, as in Example[3.4] this game can be interpreted as a game between a bank as 
the leader and an investor as the follower. The investor wants to maximize 


za), 8@ur)ar- [wear 


(equivalently wants to minimize Jp f qG(T)) + fo g(x,ur)dt + fi Yr (ur(t)) dr). 

The term —q(x(T)) represents the wealth of the investor at the final time T and the term 
— d g(x,up)dt represents the consumption during the time interval [0,7]. The term ur(?) 
can be interpreted as a density of the investor's transactions with the bank, i.e., during the 
time interval [t,t + dt] the number of transactions equals up(t)dt. For up = 0 no transactions 
take place (yı(0) = 0). The transactions cost money and we assume that the bank wants to 
maximize these transaction costs L. These costs are included in the costs of the follower 
Jr. A reasonable restriction on YL is that y_(-) is nondecreasing with respect to |up|. The 
higher the number of transactions (either buying or selling, one being related to a positive 
ur, the other one to a negative ur), the higher the costs. 

The problem as stated here is obviously a difficult one. We will consider two specific 
examples of the problem (P) and try to find the optimal yr function for the leader. 
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Example 3.9 
This example is a dynamic extension of Example[3.4] The problem is: 


X=ur, x(0)—1, 


(P1) a zr Hor zi! 0) + «f» ur (t DEN 


Ha = max f YL( up (t 


As in ExampleBB.4]the transaction costs will be first not considered. Based on the results 
of the minimization problem of the follower's function without the y_-function we will 
consider different possibilities for y_-function to obtain the best-possible outcome for the 
leader. Therefore, the follower's cost function to be minimized is first defined as 


1 
god GJ Az) (3.14) 


The Hamiltonian of the problem given by X = ur, x(0) = 1, and by is 
H = kur + m 
Using the Pontryagin minimum principle we compute that 
up(t) — —À, À —0, MI) =x(1), u(t) 2 —x(1), +e [0,1], 


and hence 


1 1 1 1 
up(t) = 5, x(t) -1-3t, Jr (up = ug) = 7, Je (ur = 0) = 5. 


Mimicking the choice of yr in Example we will first consider y, defined as 
def (1 
YL (ur) =— a E up (1 +ur) 


on the interval [0,1] and yt (ur) > 0 elsewhere, with € | 0. Here y; (-) = 0 if € = 0 on (0, 1] 
and 


1 1 
H = Nur + sup — (5-2) ur (1 +ur). 


Therefore, 


and 


Â=0, A(1)=x(1), x'(n21- at 


The outcomes of the game for the leader and the follower are 


m 
4” 8 4^ 
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The leader can do better, however, with another choice of quadratic yL. Let 


def 
) = 


1 
YL (ur 5 Buk + our, 


then 148 , 

+p—a +0 
Ne ae 
x'(1) 2+B , up(t) DB’ 


where D > —1 has to be satisfied (second-order conditions). Since Ir(ur = 0) = i. the 
parameters o and B must satisfy Jr(ur = u$) < $, i.e., 


2 
5 (u? +) exi = FE <5 


which is always fulfilled for D > —1. Consider 


a lfef(lta\? altai B-4a- Gpo 
pio. 2 ( (>) if) a AEP ae 


The maximization with respect to a leads to a = ae which, upon its turn, leads to 


1 
ras (ug) = s X43 BY 
Based upon this, the e-optimal value for B is B = —1 +£, where € | 0. Subsequently, a = 
-3 T ze up to first order in g, and with the same accuracy, up = -4 SE Je . This leads to 
Pee ae 
9 27 6 18 


which is a best result for the leader within the class of quadratic y_-functions. Without the 
transaction costs for the investor, its costs would be 


which is less than what he would have obtained by playing ur = 0. Since g (ui) > 0 in a 
neighborhood of us, only further away from u% the function yı has to be adjusted such that 
Yr (-) > 0 everywhere. 
We might think that the leader can maximize his/her profit by means of the following 
nonquadratic choice: 
0, if up=0, 
Tur) = { l-e, if up #0, 


with e | 0. It can be derived that the profit of the bank is 3 which is clearly less than what 
could be obtained with the best quadratic YL. 


To show that with use of the quadratic y -function the profit of the leader is maximized, 
we will formulate the discretized version of Example 
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Example 3.10 [Discretized version of Example[3.9] 
Here we consider the model 


and the criteria 


1 y (ul)? +3 ie ? 1 
= — u = — u =— 
JF 2N = F 2 Ne F ’ JL N: 


The expression to be minimized is then 


The derivative of this expression with respect to ul) equals zero. Substitution of ul?) = 


= ul’) = ul) subsequently leads to 


(i)\* N X 
(u) ~~ 3N-Tl - 1,2,...,N. 


With N — œ we get (ut) = —4, i=1,2,...,N. The profit for the leader is ZEN: Note 
that with limy_... ZEND = L, as in the case of the best quadratic y; -function. 


If we consider Jp as a function of ut only and with ul?) — ul) = A then Jp 


can be computed as 


1) Q N N 
n (P... d ) --x-) - 


IIIa N-I N V 1 (a) N(N-1)NV 
eG) CUN (s) OEE ave 


and 
1 2 N N 
v (uf? SS) 
E (1) Ne Nw N 
ze (uf avc qe ee s) 
_1/N(N-1)+(2N)? N+Q@N-1) 
2 GN — 1 BN-1 [ 
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the result is ZEND which equals the profit of the bank (as already obtained earlier). Apart 
from an €-term it is necessary for the leader that 


hari N (2) m) N 1 
se (u ENS 2 eo an si) N“ 
(1) Q) (N) N 
< = ron Es 
< Ir(up 0, ug D ‚Ur Ni 
or with a quadratic €-term, 
1 (1) Q) (N) N 
JF T wit u JF c a 0; TE y ME E "3N-1 


T NM NM 
“w tzai) WE AS 


GN — 1) N 


rin) in) | 


Ef My, 2N n 
(U) eed 


For N — oo this leads to exactly the optimal quadratic function obtained before. This is at 
least true for € = 0. The terms linear in € differ, however. We now write 


se (a...) = 98 (WP) 


The Hessian equals 


2€ 1 
N NZ UM 
Ca 
N? 
3 n 
N? NT N 


For N > + all eigenvalues lie in the right half plane. For N < x. however, the Hessian is not 
positive definite. Therefore, for RT < up <0, yL (ur) is as above, and for ur < - 7 
we choose it as a decreasing function. 
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Example 3.11 
Let us consider the dynamic model defined by 


X=ur, x(0)=1, 


with criterion 


1 f! 1 
min = | (x? + ug?) dt + 2x? (1). 
up 2 0 2 
An important difference with the problem of the previous section is that the optimal control 
is not constant anymore: u(t) = —e~', which leads to the minimal value 7 = i. There- 


fore, in the discretized problem (see the coming subsection) all Ga cannot be equal 
anymore. Consequently y, (ur) will have to be specified in the neighborhood of these dif- 
ferent (u) -vatues. 

We will calculate yg as in Example First we consider a y-function of the form 
YL (ur) = Bur? + Our. The value function, to be minimized with respect to & and D, is 


(assuming that x(0) = 1) 
55(0) --(0) +m(0), 


where S(t), k(t), and m(t) satisfy (see Section [2.5.3] 


08g 
ee ee, et 
1+ß : : 
m= lt m(1) — O. 
1+8 2.7 


It would be very difficult to proceed in analytical way from here, and, therefore, we will 
proceed with the discretized version of Example|3.11 


Example 3.12 [Discretized version of Example[3.9] 
The model is 
la 
Xi xi] uu i=1,2,...,N, spe, 


and the criteria are 


jm ne 1 
naQ) +) +598 = 
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First we want to solve min, Jp subject to the model equations. This leads to a linear equa- 
tion in up-elements: 


d+% & G … Cy "uU t 
Go d+b2 Ca ut b 
3 

65 63 d+% : 4) |--N| 5 |, 615 
; Cy 

Cn Cn d+Ln un ON 

where 
1 1 . 1 


The numerical computations suggest that the solution up converges towards —e~‘, as it 
should, as N — œ. An upper bound for what the leader can hope for is obtained via the 


calculation of the maximum (with respect to uÖ, i — 1,2,...,N) of 
Jr(O, ug) ue ul, pes ut) Jeu uu, BR uy 4 
mu 0u u. u) = Tu ud ud, Des PES 
Je (uU? uo uU... ul) = su uo) u). u) (3.17) 
ru? uu), TN uM 0) Jeu uu), xs ut), 


Written out this expression becomes 


1 1 N 5 1 N N 1 2 2 i—1 
(ate) E (h) Sr de (s (P) exp Y e). 


l=1 l=1 i=l+1 k=1,Al 
(Oy p) |y y2,4 Ly 0 
Lie Lu) u. b gee a 
NEA ( i=1,Al 2N tna N NT 
Differentiation with respect to ul! ) leads to 
d+% 26 265 e. 26n ul) & 
20 dtl, 2 i & 
; 3 
203 23 d+ß GN ha 5 y (3.18) 
: - 20v ` 
(N) 
26v ee ... 25x d + Cn Hp Sw 


with d and G; defined as in (3.16). If this linear system of equations is symbolically written 
as („I +A)up = —NG, I being the identity matrix, then 


up = —(I— NA + (NA)? — ---)N?E. 
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Numerical computations suggest that the solution up resembles the one of (3.15), i.e., the 
values are negative, |u(i)| decreases with increasing i. 


Above it was assumed that all ul) -values were different (which also followed from nu- 
merical evidence). In case of all u() -values being equal, an upper bound for the best out- 
come for the leader is the maximum value of ö for which the following inequalities hold for 


a choice of the @;’s (all being equal): 
Jp(0,00,..., Ow) + ni < Jp(“ a's are zero, all combinations"), (3.19) 


i — 1,2,...,N. In general the leader will have to deal with a mixture of the extreme cases 
and (3.19) since he/she will not know ahead of time what yields the best results for 


him/her. Suppose that all optimal ul? are different. Then choose 


(1) (I) 9 4D (N), 


ya (u) = ze (ul NE: GD.. uk a) (2) (3) My 


— Jg(up Up’ Up Veg) — E, 
and choose for all other values of ur Yy, large (except for y (0) = 0). In general this function 
will not be monotone with respect to |up| and its “usefulness” seems questionable for N > 


oo, 


3.2.2 One leader - multiple followers games 


In this section we will mention a dynamic problem of the inverse Stackelberg type with one 
leader and two followers. 


Example 3.13 
Let us consider the following example: 


x1(0)=0, %(0)=0, #1 =ur, up, X2—ur, 
] f! l 
Jp, = zl ug, (t) dt + 510) «f YL (ur, (t)) dt, 


1 
m= [why art o0) [n us (0) ar. 


Both Jp, and Jp, are to be minimized by the followers, while 


„Ef arr 0) 1. ar OD (3.20) 
0 


is to be maximized by the leader. 

Our problem is to find the optimal y, (-) maximizing (3.20), while 7r, and Jp, are mini- 
mized by the followers. 

Let us first consider yr (-) defined as 


Yılur) S aug? Bug, Vie {1,2} (3.21) 


With this y we can compute 
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2 2 
Hi = zug, t Our, + Bur, +A, ur, ur, 


2 2 
Ay = zug, + Auf, + Bur, + Aı ur,, 


and, therefore, 


2(1-- a) (B--2ap— 1) 
(5+ 1604 20074 803+2B +B)’ 
" 1+ß 


UR, zi 
OE) 
and 
_2(1+0) 2o? -aß?+30+ß-+1) 
5 + 160+ 2002+ 803+2B+B2 ' 


| 1420-28 — B? 

2 wks 

"E 4a(1 +a)? (B+2aß-1) 2B(1+a)(B+2aß-—1) 
u y y 


a(B+1) B(B+1) 
4(a+1) 2(a+1)’ 


with y & 5.1160 +200? +803 + 2p + ^, y # 0,1. Minimization of 7, with respect to o 
and ß gives a* ~ 0.696, D* ~ —0.111. 

Substituting & = ot“ and B = D" into JL, Ir, , and Jp, , respectively, gives us 0.472, 0.383, 
and 0.076, respectively, and 


uj, =0.151, uj, = —0.262. 
Let us now consider a linear yr (-), i.e., YL (-) defined by 
w(x) fox vx eR. 


Then using the same approach as in the previous case we obtain 


s 2 (14- o) 
n 5420-4 02’ 
1 ld b 
x o oU 
(o a(1+11+30? + ao) 
m 2(5+2a +a?) 
Minimization of J with respect to ol gives 
a* ~ —0.047 


and Ir, ~ 0.234 - 1072, Jp, ~ 0.249, and JL ~ 0.429. 
This is a slightly worse outcome for the leader than in the previous case. We could not 
find any better solution with yı(-) defined as a polynomial of higher degree than 2. 


3.3 Extension: Two leaders — one follower 35 


3.3 Extension: Two leaders - one follower 


In the following example we will study the problem with two leaders and one follower. In 
this case there is not an obvious point on which both leaders will agree at the outset. Hence 
we will not try to start with such a point. This example was introduced in [65]. 


Example 3.14 
The follower has cost function 


Jg, = ug, HUL, HU 
and the leaders have the cost functions 
JL, = (ug, - 1? + (ur, 1), I = (ur, 2)? + (un, — 1). 
respectively. Suppose that the two leaders will choose their functions uj; as 
up, = YL, (up) = Our, +02, UL, = YL, (Ur, ) = Biur, + Bo. 


In the three dimensional (ur, ‚ur, ,UL,) space these two planes have a line of intersection and 
the follower is forced to choose the best point (i.e. with the minimum value of Jp, ) on this 
line of intersection. This leads to 


_ 0400 + Bio 


u => " 
Fi 1407 + Bî 


Realizing this choice of the follower, the two leaders will substitute this choice into their 
own YL;‚-functions and subsequently into their own cost functions. Thus these cost functions 
become functions of the parameters &; and fj, i= 1,2, only. By setting 


OIL, (0,05, Bi, B2) 
do; 


OIL, (03,00, B1. B2) 
opi 

i.e. the necessary conditions for a Nash equilibrium, one obtains four equations with four 

unknowns. The solutions are 


=0, =0, 


Q;——5, O=—10, Bi =-2, f5—5, 


with corresponding up, =2, uj, —0, ur, = 1, and 


with corresponding ur, = 1, up, =1, ur, — 0. Besides some other solutions were indi- 
cated which result from the roots of a fourth order polynomial. 

Let us study the first solution given in more detail. It turns out that the second order 
conditions are fulfilled. Hence a correct solution has been obtained. It is striking that the 
resulting up-values coincide with the absolute minimum of the second leader (moreover, the 
second solution obtained corresponds to the absolute minimum of the first leader). 

It is claimed now that the solution obtained is only locally optimal. If the second leader 
sticks to 


uL; = YL, (up, ) = —2up, +5 
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it is claimed now that the first leader can do better than 
Up, = YL (ur, ) = —Sur, + 10, 


namely by playing 
uL, = YL) (ur, ) = Sur, —4. 


With this choice of y_,, while yj, remains the same, i.e., Yr, (ur) = —2ur, +5, the first 
leader obtains his/her team minimum (ur, = uj, = 1). How has this latter y, function been 
obtained? Answer: by substituting the function uj, = YL, (ur, ) = —2ug, +5 into the cost 
functions Jp, and 9,,. This now is a game for one leader and one follower. The leader can 
obtain his/her team minimum in this case. 

If the first leader chooses uj, = YL, (ur, ) = Sur, — 4, then the second leader might be 
willing to change his/her yr, function. Indeed, that is the case. With uj, = YL, (ur, ) = 
+5up, — 4 (fixed) and ur, = YL, (ur, ) = —32ur, + 65 the resulting (ur, ‚ur, ur) coincides 
with the absolute minimum of the second leader. If the leaders continue with alternately 
adapting their optimal functions we obtain: 


up, ) = —2up, +5, 
UL, = = +5 up, —4, 


Yu, (ur, ) 
YL, (ur, ) 
Up, = Yulur,) = —32ur, +65, 
Ya (ur, ) 
Ya (ur, ) 


u Ly 


up, 


up, ) = +1055 ug, — 1054, 
„(ur ) = —1114082 up, + 2228165, 


UL, = 
UL, 


etc. 


Obviously this algorithm does not converge, linear Y -functions cannot lead to a Nash solu- 
tion. 


In the following theorem (ur, I, UL) refers to the pair (up, ‚uL,) that minimizes 7[,. 
Similarly, (ur i, ; UL,,n,) refers to the pair (ur, uj) that minimizes J,,. 


Theorem 3.2 /f ur, Ju # ur, s, a Nash solution between the leaders does not exist. 


This theorem holds irrespective of the class of Yr, (ur, ) functions, i = 1,2. These functions 
are allowed to be discontinuous (even with an infinite number of discontinuity points); the 
theorem remains true. 


Proof: See 651. 


Example 3.15 
Let us consider the cost functions of example[3.14lonce more, but now with the constraints 
—l < up, € +3, -1 Sur, € +3, i = 1,2. The roles of the players remain the same. We will 
let the two leaders alternately minimize their cost functions and see whether this algorithm 
converges. 

We start by assuming Yr, to be given with ur, = YL, (ur, ) = 0 (A two-player Stackelberg 
game results with L» as leader and F| as follower). Their cost functions are respectively 


In, = ug, + ui, A, = (ug, — 2)? + (u, 1^. 
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An optimal choice for Lo is 


3 if up, Z2, 


n, = Yun) = { 1 if up, =2. mes 


As a result of this choice Fj will choose up, = 2. Subsequently uj, = 1 and Lj has realized 
his/her team minimum. Note that many other choices for yj, are possible with the same 
result, e.g. 
ee j= —2ug +5 if 1<up, <3, 
2 EN 3 if —l<up, S +. 

We will continue with the first choice for yı, given, i.e. (3.22). Keeping this function 
fixed, the other leader, Lj, will now choose his/her optimal yr, (ur, ) function. Equation 
is substituted into 7r, leading to 


NN 2 9 if up, Æ 2, 
Jr, = UF, X 1 if up, =2. 
It is easily verified now that an optimal solution for L; is 


3 if up, #1, 


a, 7.08) = { | if up, =l. des 


This leads to the team minimum of Lj. Also in this case, the optimal yr, is not unique. 

We now fix YL, as given in and study the best answer by L2. Lz cannot obtain 
his/her team minimum anymore, since Jp, prefers playing up, = 1 to up, = 2, whatever the 
choice of yr, (+). The worst that can happen to player F; is the outcome 11 which is real- 
ized for up, = 1, up, = Yr; (ur, = 1) = 3, ur, = vr, (ur, = 1) = 1. Hence Lo should con- 
sider min. u, JL, (ur, ur) subject to Jr, (ur, QUL, = Yi (ur, ), UL, ) < IF, (ur, = Lur, = 
1,u_, = 3) = 11. This leads to 


3 if up, ALE), 


u, = Yi; (ur) = | AZ 


, (3.24) 
vs & if up, -2(3$ —e1). 


as a possible choice for L2. The value ej > 0 has been added so as to make the choice for 
Fi unique after (3.24) has been announced. For ej = 0 player F; has two choices, but one 
of them is preferred by Lo. 


In this way we continue, keeping Yı, (ur, ) fixed again, the new (actually: a new) optimal 
answer by L; turns out to be 


3 if ur, Z Vl—&|v 10 — £2, 
l—e,V10—€ if up, = 1—e,V10-—€, 


for a small positive €2 such as to make the answer by F; unique. If we continue in this way, 
the algorithm converges to 


uL, = YL, (up, ) = | (3.25) 


3 ifu 0, 
YL, Qm) = Yo (ur, ) = { 0 if s E 
‚=0. 


This solution leads to the team minimum of the follower (!). Apparently the follower is "the 
laughing third party”. 


The problem introduced in this section can be extended to a situation with multiple leaders 
and multiple followers. For more information about this topic, see 165]. 


38 3 Inverse Stackelberg Games 


3.4 Conclusions and future research 


In this chapter we were dealing with one leader — one follower and one leader — multiple 
followers inverse Stackelberg games with the aim to find the optimal strategy for the leader. 

We showed problems in which the optimal strategy of the leader exists as well as the 
problems in which the optimal strategy does no exist. Problems with unknown optimal 
outcome for the leader were elaborated mainly in Sections3.2.1]and[3.2.2] In such situations 
an ad-hoc approach was used in order to find the best possible outcome for the leader. 

Since the theory of inverse Stackelberg games is still in its infancy, the important phe- 
nomena was shown mainly by means of examples. Further investigation of the properties 
of these games is a subject for future research. 


Chapter 4 


Static Optimal Toll Design 


In this chapter the static optimal toll design problem will be dealt with. Here the word static 
refers to the situation in which the traffic does not evolve over time, i.e., the problem is a 
one-shot game. In Section an overview of existing literature about the static optimal 
toll design problem will be proposed. In Section [4.2]the problem will be introduced, to- 
gether with basics from modeling of traffic on road networks. In Section [4.3]two concepts 
of drivers' behavior will be introduced and explained, together with their properties. In Sec- 
tion a problem formulation of the static optimal toll design problem will be given. In 
Section [4.5]the general problem properties will be discussed. In Section possible ways 
of how to solve the problem will be proposed. In Section[4.7]the proposed solution methods 
will be illustrated on case studies with the network with one origin-destination pair and 
on the so-called Beltway network, respectively. In Section [4.8] possible extensions of the 
research presented in this chapter together with a summary of the research of this chapter 
will be given. 


4.1 Introduction and literature overview 


The optimal toll design problem is a problem of the Stackelberg type isl lel, applied to the 
traffic environment with a road authority as a leader and travelers as followers. The aim of 
the road authority is to minimize its objective function, which is dependent on the travelers’ 
decisions, by choosing optimal tolls for a subset of links (so-called tollable links), while 
the travelers minimize their individual travel costs. Their behavior is usually modeled by 
applying a traffic assignment procedure [35.69]. 

If it is assumed that all drivers are rational and have complete and perfect information 
regarding network conditions, the deterministic user equilibrium (DUE) applies [94]. With 
imperfect information and distributed preferences a probabilistic user equilibrium, referred 
as well to as stochastic user equilibrium (SUE) takes place, for example, as a logit-based 
stochastic equilibrium (LB-SUE), see (58). 

There are two main research streams with respect to definition of the set of tollable links. 
With so-called first-best tolling (or pricing), all the links in the network are assumed to be 
tollable [es log), with so-called second-best tolling not all links are tollable [85]. The latter 
concept is clearly more applicable in practice. 
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In and second-best tolling is considered, travelers are driven by the determin- 
istic user equilibrium (DUE), the objective function of the road authority is defined as the 
surplus of the network, the traffic demand is elastic, and it is assumed that the link cost 
functions are increasing with respect to traffic flows. In lech the lower level of the 
problem (travelers’ minimization of travel costs) is formulated and solved as a variational 
inequality problem (VIP). Here the travelers are driven by DUE. In a very general 
Stackelberg model is presented, where the road authority has two decision variables, one 
of them possibly dependent on traffic flow. The paper itself deals with general mathemat- 
ical properties of traffic equilibria, however. The tolls are assumed to be constant and the 
traffic-flow-dependent variable is interpreted as management decision of the road authority. 

Following extensive case studies of two-route congestion problems in static networks 
i24 [sa sg], we have introduced its second-best variant in which the link tolls are functions 
of link and route flows in the network, for only a proper subset of all links. This formulation 
fits within a theoretical framework of "inverse Stackelberg problems" 80]. In the inverse 
Stackelberg game the road authority introduces tolls as mappings of the traffic flows in the 
network and, therefore, the possible responses of the drivers are taken into account in the 
first place, while in the classical Stackelberg game the traffic-flow invariant toll is set first 
and the drivers react as second. In both cases, the road authority is the leader. 

This chapter introduces an extension of our recent research to the general problem of 
optimal design with traffic flow-dependent second-best tolling. 

Because the problem is at least NP-hard, advanced optimization techniques, which can 
be parallelized, should be used in order to speed up the solution process. In this chapter an 
algorithm using neural networks is proposed as such an optimization technique. 


4.2 Preliminaries 


Consider a strongly connected road network G = (N , 4), with a finite nonempty node set 
N. = {1,...,n} (n € N) and a finite nonempty set a = {1,...,|a|} (|a| € N) of directed 
arcs (links). Let RS C N x N be a set of origin-destination pairs in the network. We 
denote the nonempty set of simple routes (i.e., routes without cycles) from the origin r to 
the destination s by ? (^9) and the set of all simple routes of the network by ?. Let T C A 
be a set of so-called tollable arcs (links), i.e., the links on which toll can be imposed. 

Drivers in the road network G (X, 4) travel from their origins to their destinations, be- 
ing noncooperative among themselves. When using tollable links, drivers might be obliged 
to pay a prespecified toll. Drivers choose their routes in order to minimize their travel costs. 
Each of the travel costs is a combination of travel time and tolls. The travelers' choices will 
determine the traffic flows in the network. 

There is a road authority that sets tolls on the tollable links in the network in order to 
minimize its objective function. The toll values are assumed to be calculated as functions 
on traffic flows in the network. 

Each change of the tolls will cause change in the travelers' behavior, and vice versa. 
The optimal toll design problem introduced in this chapter is a one leader-more followers 
inverse Stackelberg game with the road authority as the leader and drivers as the followers. 

There is a fixed positive travel demand described by drivers traveling from origin r to 
destination s: d ^9 [veh/h]. Let us denote the link traffic flow on link a € 4 by qa [veh/h] 
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and the route traffic flow on route p € P by fp [veh/h]. 
Feasibility with respect to the travel demands requires the route flows to satisfy the 
following conditions iid 


d= Y fa (ners, (4.1) 
per (rs) 
fo>0, perl), (rns)ems. (4.2) 


We let õp a be a link-route incidence identifier for G (N , 4) , defined as 


1, ifroute p € ? contains link a € A, 
õp a = : 
0, otherwise. 


The link flow on link a is defined through the route flows in the network by the relation 


quc Bk Fe Hill} (4.3) 


pee 


With each link a € 4 the link cost for travelers entering this link G4 [euro], defined as 


Ga (qa) = Qta (qa) +6, (qa); (4.4) 


is associated. Here & > 0 [euro/time unit] is the travelers’ value of time, ta = ta(qa) [time 
unit] is the link travel time on link a, and 0, = 0,(q,) [euro] is the toll paid by each traveler 
for using link a. 

The link travel time function is assumed to be traffic-flow dependent, continuously dif- 
ferentiable, and increasing with the link traffic flow. The standard way to define the link 
travel time function is 

IK] 
talqa) = Y, Weak, |KIENo, where y,»0 Vk. (4.5) 
k=0 


If K = 0 the link travel time is traffic flow-independent. Another very common link travel 
time function is the Bureau of Public Roads Delay Function cda, defined as 


X2 
talqa) = fa ( TX (&) ) » X1,X2> 0, (4.6) 


where t40 [time unit] is the free-flow travel time on link a, Ca [veh/time unit] is the capacity 
of link a per time unit. 
The route costs cp, (p € P) are assumed to be additive, i.e., they are derived from the 
link costs Ga (a € A) through the relation 
Gs us (4.7) 
aca 
Let q, t, and ¢ denote a vector of link flows, a vector of link travel times, and a vector of 
link travel costs on all links in the network, respectively, i.e, 
gi fi isi 
der | 92 def | 2 def | & 
q= ; , t= . y = i i (4.8) 


CN tA Sal 
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Similarly, let us define corresponing route vectors: 


fi qT C1 
ar] der | 72 der | €? 
f= . ER d y. C= 
fie] Te] Cje| 


On each link from the set of tollable links 7 the road authority can impose a traffic-flow 
dependent toll. The traffic flow-dependent toll on link a € T will be denoted by 0,(-). This 
toll is defined as a polynomial function of the link flow on the same link,! i.e., 


u 0 for aea\T 
) (m) m (m) , 
alda) ma wa Mera Wa x? fo aeT, x" ER, en 


with M € No. The traffic flows for the coming time period can be observed when playing the 
game repeatedly. If a new toll level scheme is set on the real network, in a finite time (after a 
finite number of days), during which the travelers try different possibilities of their traveling, 
the system is assumed to reach an equilibrium state. The road authority is assumed to be 
perfectly aware of the possible reactions of the drivers to a given vector of toll values and, 
therefore, can set the toll as defined in (4.9). 


By definition, 


=0 if aea\T, 
eu (a) SO. df gee. (4.10) 


This means that the drivers cannot receive rewards when using tolled links. The vector 0 
will be a vector of link toll functions” and can be symbolically written as follows: 


"M () 


Wa 
wid ui 
2 
wat | w |, w*| ^" | (4.11) 
Wal 
wi) 


'The motivation for choosing a polynomial toll function is connected with the fact that the polynomial link 
travel time functions are used in this thesis and, therefore, the first-best toll is a polynomial function of the link 
flow, too. Other option would be to map the link tolls to the link travel times, as those are best congestion measures. 

Note that a = 0 iff a € A Va. 
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Let wi) € y (n), min. , (m) max with yr) min. (m), max € R, wm) min < p(n) max for Vm € 


{1,...,M} and let sets W, and W be defined as follows: 


, 


W, def [Omrin omes x win, mes Xxx [nin man 


wt 


W,)^! vme(0,...,M). 
Clearly, W, is a subset of RV and thus W, and W are compact sets. It is assumed that 
Wa € Wa, w EW. 

With M = 0 in equation the toll becomes traffic flow-invariant. In that situation the 


toll on the link a will be set as wl) € RO, and vector w defined as 


will be a vector of traffic-flow invariant tolls. 


4.2.1 Game-theoretic interpretation of the optimal toll design problem 


The problem of the optimal toll design can be seen as an inverse Stackelberg game. Two 
possible interpretations from the game theoretic point of view are possible: 


e The drivers, as followers, choose routes from their origins to their destinations so as 
to minimize their actual or perceived travel costs. Therefore, their decisions are their 
route choices. Because the traffic flows are dependent upon these decisions and the 
road authority as the leader sets tolls as functions of the traffic flows in the network, 
these tolls are also composed functions of the drivers’ decisions. 


Because the travelers are uniform, all of them can be seen as one super-player, who is 
the follower in the one leader — one follower inverse Stackelberg game with the road 
authority as the leader. The decision of this super-player would establish the traffic 
flows in the network. The tolls are the functions of the follower's decisions in this 
game. 


In order to model the travelers's behavior (route choices), a traffic assignment model has to 
be used. In the following section we will discuss such a model. 


4.3 Drivers’ behavior — static traffic assignment 


This section formulates macroscopic static traffic assignment (STA) models that describe a 
way of how individual drivers choose their preferred route from their origin to their destina- 
tion. The basics of travels' behavior models introduced in this section can be found in, e.g., 


led). [67], or ssh. 
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The static traffic assignment contains a path choice model in which all travelers are dis- 
tributed on all available routes such that a particular static user equilibrium state is reached. 

Different variants of network user equilibria can be found in the literature, as the con- 
cept of equilibrium is closely related to the properties of the users that give rise to this 
equilibrium. Network equilibria fall into game-theoretic equilibria discussed in Chapter [2] 
and derive their properties from those of the participating players (i.e., network users), de- 
pending especially on the level of information they have about actions of other players (full 
information versus partial information or information with perception error), their prefer- 
ences (the player’s cost function), and their behavior (all players are assumed to minimize 
their own journey costs.). 

We consider a single-user class assignment, i.e., all users are assumed to affect the cost 
of the link they use to other players in the same way and users’ tastes in evaluating these 
costs are identical, although generally users’ perception may differ in a random way. The 
travel demand is inelastic and given. 

Various different network traffic equilibria exist (see, e.g., Lon. These equilibria can 
be seen as specific instances of games, differing in chosen cost functions. 

In this chapter two equilibrium concepts will be elaborated upon: The deterministic 
(Wardrop) user equilibrium (Section|4.3.1) and the logit-based probabilistic (in traffic liter- 
ature also referred to as stochastic) user equilibrium (Section|4.3.2). 

Although the stochastic equilibrium, introduced in Section 4.3.2) represents a more re- 
alistic concept of the drivers’ behavior than the deterministic user equilibrium, the deter- 
ministic equilibria are still widely used, mainly due to its computational advantages and its 
direct connection to the Nash equilibrium 161). 

Note that while a driver is discrete by nature, i.e., half of a driver cannot make a decision, 
we assume continuous traffic flows, which means that the traffic flows are interpolated by 
a continuous quantity. This could be justified by the fact that we are interested in average 
situations and (real-valued) expected traffic flows, in order to compute the optimal tolls for 
the road authority. A continuous approximation is also acceptable for the large traffic flows. 


4.3.1 Deterministic user (Wardrop) equilibrium 


The static deterministic traffic equilibrium, or Wardrop equilibrium, is based on the assump- 
tion that all road users have complete information about the prevailing traffic conditions, 
and that they choose the cheapest one among routes available, while taking congestion into 
account. The Wardrop equilibrium is defined as follows. 


Definition 4.1 (Wardrop equilibrium) 

For each origin-destination pair, the route travel costs for all users traveling between a 
specific origin-destination pair are equal, and less than the route travel costs which would 
be experienced by a single user on any unused feasible route within the same (r, s)-pair, i.e., 


fo(eo 19) =0, per, (4.12) 
Cp -s ns) > 0, p € prs), (4.13) 


where x”) takes the role of the minimal travel cost of the routes from ? 9). 
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For alternative formulations of the Wardrop equilibrium (including among others node-arc 
optimization formulation, arc-route optimization formulation, or MPEC formulation) and 
mathematical properties of Wardrop equilibria (including existence and uniqueness), see 


ledled. 


4.3.2 Probabilistuc (stochastic) user equilibrium 


The main drawback of the Wardrop equilibrium point of view is that each traveler is sup- 
posed to have perfect information about the whole network. In more realistic formulations 
each driver minimizes his or her perceived travel costs, i.e., their route valuation is subject 
to a random error term (either because we do not know their taste, their cost perception is 
flawed, their knowledge of the least-cost routes is flawed, or because they have information 
that is unavailable to the road authority). The so-called stochastic user equilibrium applies. 


Definition 4.2 (Probabilistic (stochastic) user equilibrium) 
For each origin-destination pair, the perceived route travel costs for all users traveling be- 
tween a specific origin-destination pair are equal, and less than the route travel costs which 
would be perceived by a single user on any unused feasible route. 


The perceived travel cost from Definition is defined as the sum of of the effective 
route travel cost? and a random unobserved component: 


Ga (4a; £) — Ga (qa) T £a; (4.14) 


where qa (the traffic flow on arc a) is considered as a macroscopic deterministic variable, 
and E, (£) follows some probability distribution, the same for each a € A, with parameter 
€a (expressing the perception error). 

We can distinguish between the case when the user makes the decision for his/her com- 
plete route (open-loop game, see Chapter[2), and therefore makes an error in the perceived 
cost of the route, from the case when he/she takes a new decision at each crossing (closed- 
loop game, see Chapter[2), and therefore makes multiple perceived cost errors during his/her 
journey. 

When the probabilistic error distribution of error € is known we can define stochastic 
assignments. 

The Probit assignment is an example of the closed-loop model. The errors E,(€) are 
supposed to be centered Gaussian (normal) random variables. However, the computation of 
the Probit assignment is difficult and is done using Monte Carlo methods. 

In the open-loop situation, with the independent, centered, and Gumbel* distributed 
errors on the perceived costs of the routes (not the arcs) with the same variance (see 
[sg], the probability that a single driver chooses route p € ? ^9 can be computed as follows: 


exp(—ucp) 


Er (4.15) 
Lcr (rs) EXP (—ucp) 


PÍa c, Vp#B, Bere) = 


with the perception error u. This u is used to calibrate the variance in the cost perception. 


3The perceived travel costs are additive. 


4P{g <x} = eur , where n is the Euler constant, the variance of G is n? /(6u*). The max of an indepen- 
dent Gumbel random variable with the same variances is still a Gumbel variable with the same variance. 
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The logit-based probabilistic (stochastic) user equilibrium (LB-SUE) conditions can 
then be characterized by the equations 


hs plz, «6 Vpzp, Be pts} II v(r,s) ERS, (4.16) 


with P [a, <p, VpAp, pc eich defined by (4.15). The conditions (4.16) are 


natural, according to the weak law of large numbers (see Bal, Chapter 8), i.e., if d (55) is 
large, and if the travelers act independently, then 


plz, <p, Vp#Ap, PE pec) Rd Jes 
From it follows that if the value of u is large, the perception error is small, and 
travelers tend to choose routes with minimal cost. Setting u — oo in yields the deter- 
ministic user equilibrium (see Section [4.3.1). Therefore, DUE is a special case of LB-SUE 
and algorithms used to solve LB-SUE can be used also for solving DUE. A small value of 
u indicates a large variance in the perception of travel cost, with travelers choosing routes 
with considerably larger actual travel costs than those being the cheapest. It can also be 
seen from that with u | O all routes within an (r‚s)-pair receive an equal share of the 
(r‚s)-demand. 


Remark 4.1 The reason for using the Gumbel distribution is the ease of computing the 
probability of the maximum of many independent Gumbel random variables and the shape 
of the distribution, which is close to the normal one. The correlations of the travel costs 
between the paths are not well represented by logit-based models and probit methods are an 
attempt to improve the quality of the stochastic models — see [71]. Nevertheless, logit-based 
methods seem to be the most used ones in traffic engineering. 

The logit distribution, obtained from the Gumbel distribution assumption on the per- 
ceived travel costs, satisfies a very important property, which justifies its use: A road having 
a smaller travel cost than another one has a larger probability of being used than the other 
one (see (Bil). It can also be seen from equation that for all values of u all routes 
receive flow, regardless of their travel time. These facts motivate the use of the logit model 
in our research. 

In other properties of the logit distribution (which is in fact the Gibbs distribution 
of mechanical statistics 391) are discussed. In particular it minimizes the entropy among 
all the flow distributions having the same average time. The free parameter u is a degree of 
stochasticity. 


For more information about probabilistic (stochastic) equilibria, including derivation 
of properties of Dial logit equilibrium via Gibbs-Maslov semirings and some well-known 
mathematical properties, see |58]. 


4.4 The problem formulation 


In Section |4.2.1| we explained that the optimal toll design problem is a game of inverse 
Stackelberg type, with possible two interpretations. 
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In this game, the aim of the road authority is to choose w in such a way so as to minimize 
some given objective function, which can be symbolically written as Z — Z(q(w),w). The 
problem can be stated as 


Find 
w' = argminy yw Z(q(w). v), 
(P)4 subject to (4.17) 


q —q(w) € UE(w), 
with w's restricted by condition (4.10). 


The expression q € UE (w) reads as “link flow vector q is a result of a used static user equi- 
librium (UE) model when a polynomial toll function defined by with coefficient vector 
w is used.” Here we assume that some equilibrium model, without further assumptions, ap- 
plies. The “standard” Stackelberg problem is a particular subproblem of (P), defined as 


Find 
(P) w* " argmin ‚yo Z(q(w),w), (4.18) 
subject to 


q = q(w) € UE (w). 


In the following section the properties of problem (P) will be discussed. 


4.5 General problem properties 


We will refer to 0 by its coefficient vector w and the pair (w,q(w)) will denote a pair 
containing the vector of chosen toll functions and the link flow vector. 

Note that problem (P) is a nonlinear programming problem (NLP) and has at least one 
solution if a user equilibrium of forms a compact set (w,q(w)). Also, if for any 
given w the set UE(w) is a singleton, w determines q uniquely (in general this would not 
determine the route flows uniquely, though). In this case, the continuity of q in terms of w 
will guarantee that the constrained set of (P) is closed, which implies the solution existence 
of (P) since q and w are bounded. 

However, since UE denotes a general user equilibrium, it might have multiple solutions 
in terms of q (UE (w) may not be a singleton). In this case, UE(w) is a point-to-set mapping 
of w Bal. 

The following theorem will be used to prove the existence of the solution of the problem 


(P). 


Theorem 4.2 A set-valued mapping ® from R" to R” is closed at any point of R” if and 
only if its graph is a closed set in IR” x R”. 


Proof: See (33). 


The existence of the solution of problem (P) will depend on the compactness of the 
graph of UE(w), defined as 


V (wq) € ((w,q)|q € UE(w), Vw € W}. (4.19) 
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Theorem 4.3 (Existence of a solution of problem (P)) Problem (P) has at least one solution 
if the following conditions are satisfied: 


i. For all w € W, the set UE(w) is nonempty and compact. 


ii. Let w,W € W and let q € UE(w), q € UE(w). For each € > 0, there exists 6 > 0 such 
that if ||w — w|| < 9, then 


max min ||g — || <€. 
VgeUE(W) VgeUE(W) 


iii. < is a continuous function of q. 


Proof: Let R(0,£) be an open ball with radius €. Then Y “UE (w) + R(0,£) is an open set 


containing UE (w). Let us define an other open set Z = (w:||w —w|| < 6). Condition ii. in 
Theorem[4.3]is equivalent to Uwe UE (w) C Y . Thus, under ii., the point-to-set mapping of 
UE (w) is upper-semicontinuous. Together with condition i. it implies that the point-to-set 
mapping UE (w) is closed on set W. Thus the graph Y (w,q) defined in is closed by 
Theorem[4.2] Also, under i., UE (w) is bounded for any w € W. Since W is a bounded set, 
the graph ¥ (w,q) is bounded as well. Thus, graph w € W is compact. Together with iii. 
and the fact that W is compact, we can conclude that (P) has at least one solution, since it is 
a NLP with a continuous objective function defined on a compact set. 


Remark 4.4 Condition i. states that for any w € W the travelers have to respond by at 
least one q, and that if the solution is not unique, that then the solution set UE(w) must 
be compact. Condition ii. can be roughly stated as "If two toll vectors are very close to 
each other, then their solution sets are also very close" (if w — W, then UE(w) — UE (w)). 
This is not satisfied for the deterministic user equilibrium, as shown in Example [4.1] but it 
holds for many user equilibrium models, including the logit-based probabilistic (stochastic) 
equilibrium. 

Example 4.1 (On properties of Wardrop equilibria) 

Let us imagine a one-origin-destination-pair network with two links, i.e., A = {1,2}. Link 
1 will be tollable, while link 2 is untolled. Let first & = 1, let t; be defined as t E qi -4qi + 


6q1, let t2 Ex. i.e., it is traffic-flow independent. Let 0; = 1.5 qi —2.5q1, let q2 4-q14-2. 
Then there are two possible solutions in terms of Wardrop equilibrium: 


0.5 
1. gerer 


. ana defa, def « » 
It is easy to see that with 0 = 0; = 1.5 a —2.5q1 +£ the outcomes would not be “close” to 


each other, and, therefore, condition ii. of Theorem[4.3]is not satisfied. 


Since (P) does not depend on the specific formulation of the user equilibrium, Theo- 
rem actually establishes the solution existence condition for (P) that can incorporate a 
broad range of UE models as long as the three conditions in the theorem are satisfied. 
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Theorem 4.5 (NP-hardness of problem (P)) 
Problem (P) is strongly NP-hard. 


Proof: Problem (P) is a quadratic bilevel programming problem iih. Even a linear-linear 
variant (with linear cost functions for the travelers and a linear objective function for the 
leader) of the problem with traffic-flow invariant tolls was proven to be strongly NP-hard 
miel Therefore, also problem (P) is strongly NP-hard. 


Remark 4.6 Although the solution of the problem (P) yields positive tolls (see equa- 
tion (4.10)), the toll functions may be decreasing with traffic flow on their own link, as 
shown in the following simple example. 


Example 4.2 (Toll decreasing with the traffic flow) 

Let us consider a problem on a network with three parallel links between one origin- 
destination pair (r,s), travelers driven by DUE, and the road authority minimizing the total 
travel time of the system. Let d (+s) = D > 0, let œ = 1. the link cost and time functions be 
linear, i.e., 


D=q1ı +q2 +43, 
&i—afi(qi) +O1(q1), Go = 0uo(q2) +02(q2), $3 = Atz (q3), 
ti(q1) = Biagi +81, t2(q2) = B242 + 82, t3(q3) = Bs q3 + 83. 


with d^? = D, a = 1, Bj = 1, B2 = 2, B3 = 0.05, 8; = 1.008, 8; = 0.672, 53 = 2. Then, 
the total travel time function can be computed as 


3 
Z(q1,92,93) = ) ajtj(aj) 
jal 


= 1.054? — 0.992 q1 + 2.053 — 1.328 q2 + 0.05 D? 
—0.1g1D —0.1q2D +0.1qıq2 4- 2D. 


The global minimum of Z(g1,92,93) is in qj = 0.457 4- 4.65 - 10? D, q5 ~ 0.313 +2.33. 
107? D, q3 ~ 0.930 D — 0.77 and reaches approximately —0.435 + 1.993 D 4- 0.047 D? [time 
units]. This is the best what the road authority can obtain, given the fixed travel demand 
(the so-called team minimum). 

Let us assume that the road authority introduces the tolls on links /; and / as linear 
functions of the link flows on the same links, i.e., 01(q1) =agı +b, 02(q2) =aq2 +b, with 
81(-),82(-) > O on (0,1). With DUE, relation G1 = c2 = G3 holds if all three links are used. 
It can be shown that for any D > 0.828 the team minimum for the road authority can be 
reached (i.e, q* = (q},q5,q3)’ is that optimal flow pattern for the travelers) and that in such 
case a < 0, while b > 0. 

The optimal tolls are decreasing with traffic flows on the same links, because link 3 
is untolled. Other choices of tollable links would bring toll functions increasing with the 
traffic flow on the same link. 


Similar phenomena will appear in some of the case studies in Section 
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4.6 Solution of problem (P) 


In this section we propose solution methods for solving the problem (P) introduced in Sec- 
tion 


4.6.1 Analytical solutions 


Small problems with drivers driven by the deterministic (Wardrop) user equilibrium can be 
solved analytically, as a system of equations. 


4.6.2 Numerical solutions 


With larger problems the analytical solutions become untractable. Standard numerical al- 
gorithms for solving the lower level of the problems with DUE are: 


e the Frank-Wolfe algorithm and its extensions; 
e projection methods; 

e relaxation methods; 

e the partial lineralization algorithms; 

e the column generation algorithms. 


More information about these algorithms can be found in, e.g., (66). 

To find the solution of the lower level of the problem (the resulting traffic flows for the 
choice of particular toll) with the drivers driven by the LB-SUE iterative numerical methods 
are used. The method of successive averages (MSA) has been applied to solve the lower 
level problem. In the MSA algorithm, a search direction is obtained through a stochastic 
network loading, and the step taken towards that solution corresponds to taking the average 
of all previously generated solutions, i.e., the step length in iteration k is 1/k. For more 
details about the MSA algorithm, see ; 

Solving the upper level of problem (P) (finding optimal toll functions minimizing the ob- 
jective function) with classical optimization methods may become intractable. If the objec- 
tive function of the road authority is convex, standard algorithms for convex programming 
(e.g., conjugate gradient methods, see Lal) can be used. However, in our case the objective 
function is generally non-convex, as it is usually a polynomial function of the traffic flow (In 
Example|4.3/a problem with nonconvex objective function is shown.). Therefore, advanced 
heuristic methods have be used in order to find a solution in a tractable way. We propose to 
use a a neural-networks approach. 


Example 4.3  (Nonconvexity of the objective function) 
Let us consider a one-origin-destination pair network with two links, with link 1 tolled with 
toll defined as a particular polynomial function of the traffic flow, e.g., 01(g1) «160 — 
25qi + a, and with link 2 untolled. The objective function will be defined as the total toll 
revenue of the system, i.e., Z er q1:91(q1) 21604; — 25 d + Cre This function is clearly 
nonconvex with respect to q1. 
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The solution method for (P) that we propose is a combination of the neural networks ap- 
proach for the upper level of the problem and a method of sufficient averages for the lower 
level of the problem. The algorithm will be proposed and discussed in this section. 

The concept of neural networks is closely related to the concept of supervised learning, 
which will be explained below. 


4.6.3 Supervised learning 


Let function g : R” — R” assign a vector yi € IR" to each vector x! € R”, i.e., y/ =g (x) . We 
will refer to the pair (x! ; y’) as the i-th pattern of the function g. The vector x! will be called 
the input vector (of g) and the vector y' will be called the output vector (of g). Supervised 
learning is a way to find an approximation of the function g given a set of o patterns [48]. 

An artificial neural network (ANN) can be thought of as a simple mathematical formula 
with parameters called weights [48]. The result of supervised learning is an approximation 
function g°PP with an appropriately chosen vector of weights s. The goal of supervised 
learning with ANN is therefore to find a function g?PP : R” — R”, that is approximating the 
function g in the “best way". Moreover, it is required that g?PP has derivatives of all finite 
orders in the components of x. 

There are several criteria that can be used to validate whether the function g?PP is “close 
enough" to g. In our approach the so-called validation error for each pattern (x, y^), i = 
1,2,...,0,1s minimized. 

The set of o patterns is divided into a set of t training patterns and a set of o — t validation 
patterns. For a given vector of weights s the training and the validation errors are calculated 
by f 
Y Y er -I 

(4.20) 
(e 5,5) vi)’, 


where ge and yl. k= 1,2,...,m, refer to the k-th entry of g°PP and y', respectively. The 
elements of s are optimized only for t training patterns, while the validation patterns are 
used to prevent overtraining. Roughly stated: If the training error &; becomes small with 
respect to s, while the validation error €, simultaneously grows, the ANN learns the patterns 
"by heart" and looses its interpolation and extrapolation abilities. 

An ANN is trained iteratively, i.e., €; is decreased by adaption of s, until £, increases for 
two consecutive iterations (prevention of overtraining). Note that the training stops before 
a local minimum of £; is reached. Weight upgrades siert! — sier can be calculated with 
any minimization algorithm, e.g., a first derivative method such as steepest descent, or a 
second derivative method such as the Newton's method. For the first derivative methods the 
iterative sequence 


gitertl = n +N (& (ue Vie bie) )As (& (sie) À Vse, [m ) , (4.21) 


with the search direction TAST 


implemented within FAUN? 1.0 for constrained nonlinear least-squares problems are 


and with step length n, takes place. Numerical methods 


Fast Approximation with Universal Neural networks 
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sequential quadratic programming (SQP) methods and generalized Gauss-Newton (GGN 
methods. These methods can exploit the special structure of the Hessian matrix of €; 
371. It turns out that in practice SQP and GGN methods can automatically overcome most 
of the training problems of ANN such as flat spots or steep canyons of the error function €;. 
Advantages of these methods are: 


e In comparison to common training methods a more efficient search direction As is 
calculated by use of the so-called back propagation (see [16] ). 


The step length n is accommodated during the training in contrast to common training 
methods with fixed step length. The number of learning steps is reduced significantly. 


Only £;, Vs£;, and £j, which can be computed by very fast matrix operations, are 
required. For other ANN structures, e.g., radial basis functions, an efficient code for 
Vse, can be derived by automatic differentiation. 


Maximum and minimum of each weight can be set easily (box constraints). 


The total curvature of the ANN can be constrained (prevention of ANN oscillations). 


Convexity and monotonicity constraints can be set. 


4.6.4 Solving the optimal toll design problem 


In this section an algorithm for finding the solution of problem (P) is proposed. The flow 
chart of the solution process is depicted in Figure 


Below we will describe individual parts of the solution process. 


1. Area selection 


(m),min (m),max 


Initially a set W with very low w and very high w will be chosen. The area W is 
changed depending on the outcome of the computation, the algorithm is applied recursively. 


2. Computation of sample points of the objective function 


This algorithm has two built-in optimization procedures: outer loop (corresponding to the 
upper level of the problem - the decisions of the road authority) and inner loop (correspond- 
ing to the lower level of the problem - the decision of the drivers) optimization procedures. 

In the outer loop of the algorithm a grid search is applied. In each step of the outer 
algorithm an element of w € W is selected according to the adopted grid and used as an 
input for the inner loop. In this way a grid of sample points of the objective function is 
created. 

In the inner loop the traffic assignment, including the route choice model, aiming to 
determine the user equilibrium based on the actual travel costs, is applied. To compute new 
route flow rates in each iteration the method of successive averages (MSA) is adopted on 


4.6 Solution of problem (P) 


1. AREA SELECTION 


2. COMPUTATION 
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Figure 4.1: Flow chart of the solution process 


the route flow level (see (66). Convergence of the inner loop is verified using the so-called 
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relative duality gap eer), defined as 


) Esens L pert) Gaan - nee) fm 


iter 


el (4.22) 


Here n^»). ite is the minimal route travel time for travelers departing from origin r to des- 
tination s as computed in the iteration iter. If the relative duality gaps of two consecutive 
iterations are close enough, i.e., if |g iter D — giten) < Emax, With a given small positive num- 
ber Emax, the algorithm is terminated. Note that gliter) | Ois the convergence criterium in the 
case of the deterministic user equilibrium. 


Pseudocode for computing sample points of the objective function 


(Initialization) 
Download the network G (X, 4), define RS, qs). T , travel demands, 
Emax (1 > Emax > 0); 
define u, n, m, iter = 0, W; 
set the network empty, compute e0); 
(Outer loop) 
for each w from set W and chosen grid do 
(Inner loop) (Logit-based stochastic traffic assignment) 
iter:=iter+1; 
while |g tet) — elite) | > emax do 
Compute link costs from and route costs from (4.7); 
Determine the route choices of travelers for each (r, s)-pair using (4.15); 
Update route flows using MSA; 
Compute link flows using (4.3); 
end do; 
Compute objective function Z(q(w),w) corresponding to w; 
Return w,Z(q(w),w). 


Finding the minimal objective function is in this stage of the computation not necessary, 
since the minimization of the functions given by the best-approximating neural network will 
take place. Note that in Section[4.6.3]the input of the neurosimulation was vector x € IR", 
which is in the following section replaced by vector w of coefficient vector. It is an || - M- 
vector. Similarly y is replaced by Z(q(w*),w*), which is a scalar. 


4.6.5 Application of FAUN 1.1 simulator 


The grid search produces the values of the objective function at discrete positions in the 
parameter space. However, the grid search is relatively time consuming. It is desirable to 
have a function that can be evaluated instantaneously. Furthermore, for every not calculated 
position in the parameter space the algorithm has to be recomputed. It would speed up the 
analysis, if the objective function could be computed for arbitrary values of the parameter 
space. This leads to the following procedure, using only a limited number of sample points 
and using neurosimulator FAUN to extrapolate the objective function by functions, that can 
be easily minimized. 


Pseudocode for applying ANN to the objective function 


(Initialization) 


4.7 Case studies 55 


aj 


Figure 4.2: Network with one origin-destination pair and multiple links. 


Prepare the grid search data for use with FAUN by splitting input and output; 
Set appropriate scaling parameters for the data; 
Set number of ANN to train successfully Sy; 
Set appropriate worst accepted validation quality; 
Prepare FAUN for parallel computation. 
(FAUN training)(Finding appropriate ANN) 
do N-times in parallel 
Select random s; 
while £, in does not grow for two consecutive steps do 
reduce €; in by following the gradient descent on s in (4.21); 
end while 
if € is acceptable 
return and save s; 
else if 
reinitialize s; 
end if 
end do 
(Postprocessing) 
Export the best ANN; 
Minimize the objective function approximation; 
Return w*, q(w*), and Z(q(w*),w*). 


4.7 Case studies 


In this section case studies illustrating the solution methods introduced in Section[4.6]will be 
presented. In Section[4.7. I|the static optimal toll design problem with a network consisting 
of one origin-destination pair and multiple links is considered, starting with two links and 
linear link travel time and linear toll function, and proceeding to the problem with more 
links and quadratic link travel times/tolls. The road authority minimizes the total travel 
time of the system or maximizes the total toll revenue, the drivers are driven by DUE. The 
problems are solved analytically. In Section a larger problem on the so-called Beltway 
network is considered and solved using the algorithm introduced in Section 


4.7.1 One origin-destination pair with multiple parallel links 


Let us consider the network with one origin-destination pair as depicted in Figure[5.T]con- 
sisting of || > 2 directed parallel routes (links). 
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The road authority as the leader sets link tolls as follows: The link ają; is untolled and 
each other link is tolled with a toll defined by (4.9), i.e., 


0 fo aea\T, 


M 
(m) m (m) 
Oa (qa) = a > Wa = nm 
kde) E" n E | xem fo aeT, kK ER. 


Travelers as followers are assumed to have complete information about the network condi- 
tions and therefore they are driven by the deterministic user equilibrium defined by Defini- 
tion[4.1] The link travel time function for a single driver traveling on a link a and the link 
travel cost function are defined as 


def def 
ta(qa) = Bada + da; Ca(qa) = Ta (qa) + 8a, 


respectively. Therefore, 
Ca (qa) = a p, dat Q Ôa + 0, (qa). 


We assume a positive inelastic traffic demand d — d (n) 0 [veh]. 

If the process of solving equations that define Wardrop equilibrium leads to negative 
traffic flows on some links, the link traffic flows on these links can be set to 0 and we 
might try to solve the new problem problem without some of these links. As shown in 
Example such a problem does not need to have a solution. 

In Example a situation with nonunique Wardrop equilibria can be seen. 


Example 4.4 (Nonexistence of the Wardrop equilibrium solution) 
Imagine the game on a three-route (link) network with one origin-destination pair. Let 
dal) 55 [veh], & = 10, and let the route (link) cost functions be given by 


45 
a---2a, (4.23) 
35 8 
LN ue 4.24 
9-7 +32: (4.24) 
5 
c3 — 104 zB (4.25) 


Then the system of equations 
c1 = C2, C2=C3 


leads to the following link traffic flows: 


T" CN MER. 
qı = 44° q2 = 44’ qa = ll 


Since q» and q3 are negative, link 2 and/or link 3 will receive zero traffic flow. Therefore, 
qı = 5, q2 = 0, q3 = 0 would be the first candidate for the Wardrop solution. If we set only 
q3 to O and solve equation cı = c3, the resulting traffic flows would be q1 = D, q2 = 3, 
which is the second candidate for the Wardrop solution. If we set only q» to 0 and we would 
solve equation c; = c3, the resulting traffic flows would be qı = 0, q3 = 5, which would 
determine the third candidate for the Wardrop solution. Unfortunately, none of traffic flows 


combinations 


e {q1,42,43} = {5,0,0}; 
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e (41,432.43) = (19, 7,0}: 
e (41,4243) = {0,0,5}. 


is in Wardrop equilibrium. 


Remark 4.7 Note that in Example[4.4lone of the route (link) cost functions was decreasing 
with traffic flow on the same route (link). The Wardrop equilibrium is a reasonable concept 
only if the route (costs) are increasing with actual traffic flows. Therefore, Example is 
of academic nature only. 


Total travel time minimization on a two-link network with linear link travel time func- 
tions 


Let us first assume that there are only two routes (links) in the network, i.e., |a| = 2, where 
the travel demand d^? is fixed. Let only link 1 be tolled. If the road authority minimizes 
the total travel time of the system, the optimal toll design problem can be written as 


Find 
01(-) = argming, () qt, 
(P1) subject to the Wardrop equilibrium constraints, 


q= (91,92) = (q,D-qı) ,t = (tt), 
and )7 4,44 =D. 


ad def def 
ti = Bigi +81, t2 = Bags +82. 

Since 2 

d" (q'-t(q)) 

——=2ßı+2ß>0, 

er 
the objective function is convex for all q. The total travel time function is minimized for 
. 2 — ô — 2 B2D 5; — -2piD 


4-7 73848) ^ 9- B+ By) Den 


With this choice of qı and q2 the total travel time minimum becomes 


4BiB2D? + (48182 +4B2ô1) D — (82 — 81)” 
4 (Bi + B2) : 


Different strategies for the road authority — An ad-hoc approach 


Let us first assume that the road authority sets toll on link 1 as a linear toll function, i.e., 


01 (q1) E we + wij qı. 


If a Wardrop equilibrium with qı and q2 defined by (4.26) applies, an optimal wO has 
to satisfy 
(001—952 — 2B2D) (1) 


Ox Ls — 
wi = 539(8» 91) + XB Be) Wi ^ 
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while wij is free. Therefore, a linear inverse Stackelberg strategy satisfying 
8; — 6; — 2B2D 
( 2=2P2D) , DER 
2 (Bi + B) 
(1) 


yields the optimal flows. Note that setting w; ^ = 0 leads to the optimal Stackelberg strategy 
0; — 50(82 — 64), which is independent of the travel demand D. 

Obviously, in this simple case there is no necessity to try more complicated strategies for 
the road authority, since the minimal total travel time can be obtained with the Stackelberg 
strategy. 


1 
81 (gi) = zu — 81) + (4.27) 


Total travel time minimization with linear link travel time functions and multiple par- 
allel links 


In the more general case, with |A] > 1 and 0,4, = 0, the total travel time function has the 
form 


A|-1 


q t= jn Gala + Qala 
a=1 


-Y« a(Ba qa + 8a) + (D — Eo (ss (2- PE i) taa) 


4|-1 |a|—1 
= Y, Baa; + Saqa + BD? — 2 Bu D 2 qa + 94D 
a=1 a=1 
lal-ı \2 lal-ı 
+ Ba | Y a) —-ó4 Y, da- 
a=1 a=1 
The optimal route (link) flow q; (a € (1,...,|A| — 1}) for a total travel minimum to be 


reached has to satisfy 


» Sjal — da P 


= 4.28 
qa 28, + 2B) (4.28) 


and gi, = d? — Yi qz 

If 8; > 64 — 2B \4)D for any a € A, the q% would be negative, and, therefore, the global 
minimum of the total travel time cannot be reached and one can try to get as close to the 
optimum as possible by trying different toll strategies. 

If õa < õa — 2Bja D and c; = Ot, + 0, fora € {1,..., Al}, expresses the link 
flows minimizing the total travel time of the system. The Wardrop equilibrium in terms of 
costs yields the following expressions for 0, in terms of 0; (a,j € A, a # j): 


ye 
EEB) (Ba Ba) 


with Y7 J- aBa (Bap j + 2B% 8; — 8a) + ap^. B;(8; — 2puD + 9j — 26,) + ABa Bia 
(28; — 94 — 8a +2Bj4)D). Since one of the links is untolled, the tolls on all other links 


90, = jea. (4.29) 
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can be computed by induction. Therefore, if y 20 Vjec A the system of linear equa- 
tions yields the Stackelberg solution of the problem. 

To recapitulate, if 8; < à — 2BD for Vj € A and V?" > 0, (a,j € A, a # j) the 
Stackelberg game with one link untolled and traffic flow-invariant toll leads to the optimal 
total travel time value. In the other cases, the outcomes of the Stackelberg and inverse 
Stackelberg game may differ. 


Total toll revenue maximization, with two parallel links and linear link travel time 
functions 


Clearly, qı maximizing the total toll revenue has to satisfy 


dd, (q1) 
—— +9,(g1)=0 
d iai) 
and, therefore, it is dependent on the definition of the toll function. We will attempt to 
maximize the total toll revenue with different toll function definitions. The problem to be 
solved can be symbolically written as 


Find 

(Pa) 0; (-) =argmaxe,(.) (q1 91 (41), 
subject to the deterministic user equilibrium conditions, 
with q = (q1,42). ,t = (t1,t2)" , and X2- qa =D. 


Different strategies for the road authority - An ad-hoc approach 
( 


With 01 (q1) = wO + wij q1 the objective function is concave for wij <0 (Erea) = 

1 
2w). Therefore, the optimal toll has to be decreasing with the traffic flow. Maximization 
of the total toll revenue function with respect to qı leads to the optimal qı and q2 (indicated 
by superscript *) 


NO „© 


* 1 * 1 
A= m DD (4.30) 
2,7 2,0 


If a Wardrop equilibrium applies, i.e., if c1 = c2, wo from (4.30) has to satisfy 


2a. (8 — B D — 8j) w\ 
y eS ep om ES Deis (4.31) 
aßı —w; c «pz 
With this choice of wo the total toll revenue reaches 
? (8) — 81 + BD)? wil) 
_ 2 (à —81 + BaD)" w) (wi? <0). (4.32) 


2 
(a: + Ra) = D 
It can be shown that the optimal Stackelberg toll is 


0; = a(dı — 82 — B2 D) 
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and that this toll yields the total toll revenue of 
a (85 5, + BoD)? 
4(Bi + Bo) 


The Stackelberg toll pays off if this expression is positive. 
Expression (4.32) has a higher value than expression if wid lies in the interval 


(6; 2-2 /1—61) (Bi - B2)a. (61i - 2-2 /1— 61) (Bi + B2): 
$1 ' 1 
with 0; = 62 — 61 + B2 D (1 > 6, has to be satisfied). 
(1) (2) 


If the toll is defined as a quadratic function, i.e., 61 (q1) = wO twi qi wi qi, the 
objective function is concave for wi > —3w2 q1. The only candidate for optimal q; is then 


(4.33) 


, 


(1)y2 (0) (2) 
" ap EN In (434) 
3wi 
With qı defined by and if the Wardrop equilibrium applies, the total toll revenue 
function has the following form: 


(wO) (2wf +31) (Dif! +6aw Ws — A wi) 
Ee (4.35) 
27 (rows +2; wij — wO wt?) 


with V, = ab, Ya = Bı + Bo. It can be seen that the total toll revenue in the quadratic toll 
case (4.35) reaches higher values than the total toll revenue in the linear toll case (4.32), if 


wi € (win. Der) with 
(1) max (16 (wP) — 609; (vf? + 27H 0!) +543 + ws) wow) 
Wi j LAN a CN a ZU MEE 


2 
27a (—8; + B2 +82)? em - wi”) 


5 
2Wowt 4/2 (2wi +391) wi? 
+— s 
27a (—8; +B2 +82)” (—»(? +2) 


where W3 = 5403 (x — 324B28182D — 162 82°) ‚and 


(1),min (16 (w)3 — 6o wi (w) + 27820) +5483 + ws) Wow?) 
Wi i LaNa ee ee MEAN 


2 
270 (—8 + Ba 4 5) ( w 4 2, D) 


5 
24/2 (2wP +31) ww” 


-——  . . 
27a (—81 + B2 +82)" (^^? -2wD) 
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Remark 4.8 The previous example suggests that in the optimal toll design problem defined 
on one origin-destination pair on the network with 2 parallel links, linear link travel time 
functions, and with the road authority maximizing the total toll revenue of the problem, the 
higher degree of the toll polynomial function yields a better outcome for the road authority. 


Total toll revenue maximization on the network with two parallel links and quadratic 
link travel time functions 


Let us consider a two-link problem with the road authority minimizing the total travel time 
of the system and the link travel time functions defined as 


def 
ta = Ba qì + Yada + õa, 
with the link toll function defined as in the previous case study, i.e., 01 (q1) = wO + wid qi. 
Following the same procedure as in the previous case it can be seen that when the Wardrop 
equilibrium holds, two possible solution flows can be reached (if the transfer condition is 
satisfied): 


(1) 
+ayı tap+2aßD- VF 
jen ayı + o yo + 2a B» 3 (4.36) 
20 (Bi — B2) 
(1) 
+ayı +a +208.) + Ps 
= qe rà ayı tay. +20 B5D + 3 (4.37) 
20 (Bi — B2) 


EU (1) (0, 22 
with % = w;^ + w) (20% +2ay -4ap5D) + wi (—4api- 405) + AV + 


20? +40 y B5 D + o2? --4 o2 Byyo D +4 02 B1B5 D? — 4 0216; 4-40? B1» 2-4 02 B6, 
— 40? B282 (V5 > 0 is a necessary condition to obtain the optimal traffic flows). 

If only one from the traffic flows and leads to the Wardrop equilibrium, 
minimization of the total travel time function gives us: 


we = 70 (282 - 28i - D), wij = ZN +2) «0. 


Note that this solution is unique. Also, since & > 0, y; > 0, and yo > 0, the optimal toll will 
be decreasing with traffic flow, provided that condition holds. A necessary condition 
for this is that 26; — 26, + y > 0. Moreover, since wid” # 0, the inverse Stackelberg game 
strategy brings a better outcome for the road authority than the Stackelberg strategy. Since 


the problem is a second-best problem, link 2 is untolled and, thus, no other possibility for 
the road authority to get the same outcome with the Stackelberg strategy exists. 


The total toll revenue maximization with multiple parallel links and linear link travel 
time functions 


The total toll revenue function has the form 


|a|—1 


g ey qug (4.38) 


a=1 
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15,16,34,35 

4,32 
9,13,25,37,41,53 
18,19,20 

10,12 

11 

others 


Table 4.1: Coefficients of the travel time function. 


2 
Here the toll 0, (-) is a function of the link traffic flow on the same link. If A + Pelaa) < 


0, then the the local maximum of the total toll revenue function is reached for qa (a € A) 
satisfying 


d Os (qa) 
— =0. 4. 
, +0, (da) ( 39) 


Therefore, the structure of the toll functions will influence the possible outcome of the game 
and, therefore, no claims on optimal strategy for the leader can be made before the structure 
of the toll function is known. 


4.7.20 Beltway network 


Let us consider the network depicted in Figure with 21 nodes and 56 links. Note that in 
Figure 4.3] link labels lie on the right-hand side of the driving direction, when going from 
North to South or from West to East. Nodes {1,2,3} will be referred to as to the North 
nodes, similarly nodes {4,9, 14} will be referred to as to the West nodes, etc. Initially the 
set of tollable links will be defined as T = {9, 11, 13, 19, 22, 23, 25, 53}, as these are in 
this network the rather congested links, when the toll is not imposed. ® 

Let the logit-based stochastic equilibrium apply for the lower level of the problem. The 
set of origins R contains nodes from North, East, and West, while the set of destinations 
S comprises nodes from the South. Let RS = R, x S. Therefore, there are 27 origin- 
destination pairs and 1357 routes in the network. There is a traffic demand of 20 cars 
for each origin—destination pair and each minute. 

The link travel time functions will be defined as tg = Ba + Ya qa + 9a q2 foreacha € A, 
where coefficients B, y, and 6 are depicted in Table[5.23 

The road authority minimizes the total travel time of the system. 

The following four problems will be dealt with considering both objectives: 


la) Stackelberg game (defined by (P.0)) with wo,4 = wo € RO, VaeT. 


6Our research does not deal with establishing the optimal set of tollable links. Research on this topic can be 
found in, e.g., Bd. 


4.7 Case studies 


WEST 


SOUTH 


Figure 4.3: Beltway network. 
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1b) Inverse Stackelberg game (defined in (P)) with M — 2, toll defined by (4.9), and with 
the additional condition that wy” = wm € R fora € T ‚Le, the linear toll functions 
have link-independent coefficients. 


1c) First-best (FB) pricing problem,with toll defined as the marginal link travel time, 
On (qa) = dd ga for each link a € 4. This presents the best possible outcome of the 
game if all links are tollable. 


1d) Problem with no toll. 


To solve these problems the algorithm introduced in Section with Emax = 1075, 
€, = 0.005, 7500 training patterns, and 2500 validation patterns will be applied. 

Note that in 1a) and 1b) the toll functions are identical for all tolled links. It is possible 
to apply more flexible tolls, but the computations of the optimal tolls would become very 
lengthy in that case. 

Let the road authority minimize the total travel time of the system, i.e., 


Z(a(w).w) = (f (a(w),w))* «(f (a(w),w)), 


The outcomes of the game with no toll and the game with the first-best tolling are de- 
picted in Table while the outcomes of the Stackelberg game together with the outcomes 
of the inverse Stackelberg game are depicted in Table 

The optimal toll value for the Stackelberg game is rather high, i.e., 38.2 euro. The opti- 
mal values of wo and wı for the inverse Stackelberg game are 0.083 and 4.57, respectively. 
The total travel time obtained with the first-best tolling is 1.3286 - 10^ [min], the total travel 
time with use of the inverse Stackelberg game is 1.3698 - 10^ [min], and 1.4435 - 10^ [min] 
with use of the Stackelberg game. The total travel time with no tolls is 1.6025 - 10* [min] 

Note that to obtain the first-best outcome it is necessary to toll 56 links, while we used 
only 2 parameters in the toll function to obtain a result rather close to the first-best outcome. 

In Table[4.4]outcomes of case studies, that differ in the set of tollable links, are depicted 
(as Case studies 2-6). These case studies are chosen such that some of the tolled links have 
very low first-best toll values, thus they should not be tolled in practice. Although outcomes 
of the Stackelberg game and the inverse Stackelberg game are quite close to each other, 
the inverse Stackelberg game performs never worse than the Stackelberg game. This is no 
surprise, since the Stackelberg game is a special case of the inverse Stackelberg game. This 
becomes clear in Case study 5 in which the optimal inverse Stackelberg strategy is in fact a 
Stackelberg strategy. 

In Case study 3 the optimal tolls on tollable links are decreasing with the link flows on 
the same links. With increasing traffic flow on the particular tolled link the other links in the 
network become even more congested, that is why the travelers should still be stimulated 
to leave the other congested links and switch to the tolled link. This phenomenon would 
not occur if the links that are more sensitive to congestion would be tolled. Therefore, the 
flow-dependent tolling can accommodate to the new traffic conditions. 

For Case study 1 additional computations with Stackelberg and inverse Stackelberg 
games were performed. We compare the following games: 


e Stackelberg game with 
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FB uli 
ame 


a [iW iow 
1,2,3 = 
4,32 
5,8,14,17,21,24 
6,7 
9,13 
10,12 
15,16 
18,20 
22,23 
25,53 
26,27,28 
29,30,31,a45 
34.35 
37,41 
38,40 
39 
47 
33,36,42,49 
43,44 
46,48 
50,51 
52,54,55,56 
11 
19 


0.16 


60.00 


0.4483 50.72 


0.16 
0.24 
7.09 
0.66 
0.72 
7.36 
4.18 


60.00 
26.40 
65.95 
81.20 
50.20 
89.88 
36.52 


1.7938 29.18 


0.55 
0.00 
0.56 
0.01 
0.56 
0.27 
0.01 
0.00 
0.47 
0.01 
0.03 
0.00 
3.07 


10.55 


180.00 
0.00 
43.35 
0.13 
10.06 
5.89 
0.01 
0.00 
8.87 
0.08 
0.90 
0.00 
31.98 
108.76 


5.16 
2.99 
5.16 
3.17 
13.37 
1.60 
3.45 
7.13 
12.70 
10.26 
5.50 
10.00 
3.35 
9.00 
10.45 
10.24 
10.00 
10.00 
10.38 
10.00 
10.03 
10.00 
4.94 
8.82 


Table 4.2: Link parameters - Stackelberg game (Case study 1) 
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— links 9,11,13,19 tolled with identical toll and links 22,23,25,53 tolled with 


identical toll, while these two toll values may differ; 


— links 9,11,13 tolled with identical toll, links 19,22,23 tolled with identical toll, 
and links 25 and 53 tolled with identical toll, while these three toll values may 


differ; 


— links 9,11 tolled with identical toll, links 13,19 tolled with identical toll, 22,23 
tolled with identical toll, and links 25,53 tolled with identical toll, while these 


four toll values may differ; 


e inverse Stackelberg game with toll set as in equation 


— with M = 1 and the identical polynomial toll imposed on all tollable links; 
— with M — 2 and the identical polynomial toll imposed on all tollable links; 


— with M — 3 and the identical polynomial toll imposed on all tollable links. 


Results are shown in Table It is clear that when comparing Stackelberg and inverse 
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1.2,3 
4 


5.8,14,17,21,24 


6,7 
9,13 

10,12 
15,16 
18,20 
22,23 
25,53 


0.00 
0.00 
0.00 
0.00 
38.20 
0.00 
0.00 
38.20 
38.20 
38.20 


flow 
60.00 
61.76 
60.00 
26.15 
48.13 
129.05 
94.64 
92.50 
19.18 
12.35 


time 
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CU CE 
a wi 


toll flow 
0.00 60.00 
0.00 59.20 
0.00 60.00 
0.00 24.81 
10.10 66.61 
0.00 106.86 
0.00 81.11 
11.13 79.03 
6.80 26.90 
6.12 18.71 


time 


32 0.00 
26,27,28 0.00 
0.00 
0.00 
0.00 
0.00 


61.76 
180.00 
0.00 
62.79 
6.15 
14.39 
13.27 


0.00 59.20 
0.00 180.00 
0.00 0.00 
0.00 62.11 
0.00 0.13 
0.00 16.04 
0.00 15.40 
0.00 0.01 
0.00 0.00 
0.00 9.28 
0.00 0.04 
0.00 1.42 
0.00 0.00 
4.64 0.79 
15.28 129.04 


29,30,31,45 
34,35 
37,41 
38,40 
39 0.00 
47 0.00 0.01 


33,36,42,49 
43,44 
46,48 


0.00 0.00 
0.00 10.16 
0.00 2.31 
50,51 0.00 7.03 
52,54,55,56 0.00 0.00 
11 3820 0.01 
19 38.20 155.69 


Table 4.3: Link parameters - inverse Stackelberg game (Case study 1) 


Stackelberg games with the same number of parameters to be optimized, the inverse Stack- 
elberg game performs never worse than the Stackelberg game. Already with 3 parameters 
the resulting total travel time is very close to the first-best outcome. Therefore, it is prof- 
itable for the road authority to calculate the tolls using inverse Stackelberg strategy even 
when the tolls are set as very simple functions of link flows. 

The average computational time with 16 microprocessors was 9.5 minutes for problems 
with one parameter to optimize, 16.2 minutes with problems with two parameters to be 
optimized, 25.5 minutes with problems with three parameters, and 40.3 minutes with 4 
parameters. The computational time can be reduced with use of more microprocessors. 


Discussion 


In the presented case studies we suggested how to improve the system performance with 
use of so-called traffic-flow dependent tolls. It can be seen that the system performance 
improves even with use of very simple toll functions. 
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Table 4.4: Results of the case studies. 
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4.57,0.083 
4.57,0.083 
4.57,0.083 
4.57,0.083 


4.57,0.083 
4.57,0.083 
4.57,0.083 
4.57,0.083 


1.3698 


ISG 

wo, W1,W2(-10— 
3.27,0.045,5.12 
3.27,0.045,5.12 
3.27,0.045,5.12 
3.27,0.045,5.12 
3.27,0.045,5.12 
3.27,0.045,5.12 
3.27,0.045,5.12 
3.27,0.045,5.12 


ISG 


wo, w1, w2 (1072), ws (- 


2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 
2.81,0.036,3.55,1.22 


1.3401 


Table 4.5: Comparison of different tolling strategies on Case study 1. 
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Another question to be discussed is the practical relevance of the proposed concept of 
the traffic-flow dependent tolls. One of the possibilities of how to apply such tolls in practice 
is to use global positioning systems (GPS) and/or mobile phones for counting the number 
of cars using specific roads/links in order to compute the traffic-flow dependent tolls. 


4.8 Conclusions and future research 


In this chapter we have introduced the problem of static optimal toll design with second-best 
traffic-flow dependent tolling. We have discussed existence of solutions of a very general 
version of this problem as well as its difficulty and we have proposed a solution algorithm. 
In the case studies (with specific objective function for the road authority and specific traffic 
assignment) we have shown both problems solved analytically and problems solved numeri- 
cally using the proposed algorithm. Some unrealistic assumptions were considered, though, 
especially inelastic travel demand. 
The following topics are subject of our future research: 


e Alternative objectives of the road authority Although problem (P) was defined 
in a general way, in the presented case studies the objective function of the road 
authority was defined as a total travel time or as a total toll revenue of the traffic 
system. Another option is to define the objective function as the reliability of the 
network or, for example, as a surplus of the network. 


e Elastic demands The traffic demand is assumed fixed. The traffic-flow dependent 
tolls can be implemented also in systems with elastic traffic demands. More about 
elastic traffic demands can be found in, e.g., isd. 


Heterogeneous network users The drivers in the network formed a homogeneous 
group. In possibilities for defining heterogeneous users are investigated. These 
possibilities can be incorporated into our problem, too. In that case different toll 
functions would be imposed for different user groups. 


The problems closely related to the research conducted in this chapter, but falling out of 
this frame, can be listed as follows: 


e Finding the best way how to model link and route traffic flow, time, and other link 
and route traffic variables. We adopted standard methods used in the traffic field. 


Validating of standard ways used to model the traffic variables on the road networks. 


e Defining criteria of efficiency of algorithms for solving the problems dealt with in 
this thesis and comparing different algorithms with respect of such criteria. We tried 
to develop algorithms that would solve the problems we are dealing with and that can 
be parallelized. We do not consider any other criteria, like speed and efficiency of the 
algorithms. 


Finding the best possible toll functions minimizing the objective function of the road 
authority. We were looking for polynomial toll functions improving the system per- 
formance remarkably when compared to outcomes obtained with standard uniform or 
time-varying tolls. 
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e Definition of the best possible objective function for the road authority from the prac- 
tical point of view. While in Chapters[4]and[5]this objective function was not defined 
and the properties of the problems were discussed with a general objective function, 
in case studies we used the total travel time (to be minimized) or total toll revenue (to 
be maximized) as objective functions of the road authority. We are aware of the fact 
that other objective functions might be more realistic. 


e Definition of the best possible way how to define travelers' cost functions. In this 
thesis the link cost function was defined as a linear combination of link travel time 
and link toll. There exist other ways how to define the link cost function. Finding of 
such ways is beyond the scope of this thesis. 


Chapter 5 


Dynamic Optimal Toll Design 


This chapter extends the outcomes of Chapter[ä]to the situation in which the problem evolves 
over time. We then talk about a dynamic optimal toll design problem. 


5.1 Introduction and literature overview 


There are extensive studies focusing on the static optimal toll design problem, i.e., on prob- 
lems in which decisions of the players (the travelers and the road authority) do not evolve 
over time (see [s5], Chapter[4). Although the static models are still widely used, the 
theory and practice of dynamic models have evolved significantly over the last ten years. 
In the dynamic version of the optimal toll design problem the dynamic traffic assignment 
(DTA) applies ( Lon. DTA models typically describe route choice behavior of travelers on 
a transportation network and the way in which traffic dynamically propagates through the 
network. 

If all travelers are assumed to have perfect information (i.e, they know the current and 
future conditions on the network as well as the decisions of the other travelers) and if they 
are uniform, the deterministic user equilibrium (DUE) applies (|10, |94]). Similarly, with 
imperfect information and distributed travel preferences, a probabilistic user equilibrium, in 
the traffic literature referred often as to a stochastic user equilibrium (SUE), applies, in the 
case studies of this chapter this is often the logit based stochastic equilibrium (LB-SUE), 
see css. 

With respect to possible tolling strategies there are two main research streams differing 
in the definition of the set of tollable links. With so-called first-best-tolling (or first-best 
pricing) all the links in the network can be tolled desl esp. With the so-called second-best 
tolling not all links are tollable (see [85]). The latter concept is clearly more applicable in 
practice. 

Dynamic congestion pricing models in which network conditions and link tolls are time- 
varying, have been addressed in Bl, where the effectiveness of various pricing policies 
(time-varying, uniform, and step tolls) was compared as well. Only one bottleneck or a 
single origin-destination network was considered there, while the possibility of applica- 
tion of traffic-flow dependent tolls is not discussed here. In and dynamic marginal 
(first-best) cost pricing models for general transportation networks were developed. As indi- 
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cated by the authors, the application of their model is limited to destination-specific (rather 
than route or link-based) tolling strategies, which might complicate its practical application. 
Moreover, only the first-best pricing is considered here. 

In the dynamic optimal toll design problem is considered with a case study on 
the so-called Chen network. Tolls are assumed uniform or time-varying, but traffic-flow 
invariant, and the problem of finding the optimal toll is defined, but not solved, although 
the impact of some specific toll values on travelers’ route and departure time choices is 
presented. 

In and second-best tolling is considered, travelers are driven by the determin- 
istic user equilibrium (DUE), the objective function of the road authority is defined as the 
surplus of the road authority (i.e., amount of money that the road authority receives by im- 
posing tolls minus the investments of the road authority concerning the toll charge), the 
traffic demand is elastic, and it is assumed that the link cost functions are increasing with 
respect to traffic flows. In and the lower-level of the problem (travelers' mini- 
mization of travel costs) is formulated and solved as a variational inequality problem (VIP). 
Here the travelers are driven by DUE. In [68] a very general Stackelberg model is presented, 
where the road authority has two decision variables, one of them possibly traffic-flow de- 
pendent. The paper itself deals with general mathematical properties of traffic equilibria, 
however. The tolls are assumed to be constant and the traffic-flow dependent variable is 
interpreted as a management decision of the road authority. 

This chapter proposes an extension of our research in the field of the static optimal toll 
design problem to the dynamic problems with both DUE and SUE. Although some au- 
thors consider the step-wise second-best tolling, to the best of our knowledge no 
research dealing with the optimal toll design problem with the second-best tolling, the trav- 
elers driven by LB-SUE, and the aim being to find optimal toll defined as a function of the 
traffic flows in the network has been done before. Since this problem is NP-hard, advanced 
optimization techniques, which can be parallelized, should be used to solve it. Similarly as 
in Chapter[4]a neural network-based algorithm as such an optimization technique is imple- 
mented. The neurosimulator FAUN has already been employed to solve other problems in 
the domain of dynamic games (13189. og]. 


5.2 Preliminaries 


Let G = (96, A) be a strongly connected road network, that means, there exists at least one 
path connects each (r, s)-pair, where X and A are finite nonempty sets of nodes and directed 
arcs (links), respectively. The set of tollable arcs will be denoted by 7T C A. There is a finite, 
nonempty set of origin-destination pairs RS C N, x N. and let the set K = {1,2,...,|K|} 
be a time index set. Here each k € K refers to 


e the interval [(k — 1.5)A, (k 4- 1.5)A) if k > 2, 
e the interval [0,0.5A) if k — 1, 


where A [h] is the length of each time interval. 
For an ordered pair of nodes (r,s) € RS, where r is an origin and s is a destination, 
there is a positive number of drivers traveling from r to s and departing during the k-th time 
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interval — so-called travel demand d^» [veh/h].! The travel demand is for the sake of 
simplicity assumed to be time-interval varying but traffic-flow and toll invariant.” Let d (ns) 
be a |X |-vector of all travel demands from r to s for all time intervals, i.e., 


, 


grs) def (arn ACD, ard) a 


Let (rı,sı) denote the first origin-destination pair, (r2,s2) denote the second origin-destin- 
ation pair, etc., let (r & 5| si) denote the last origin-destination pair. Then, 


dees) 
as l 
diss) 
In the following text we will denote such a vector by (a a" as will be a |R5|-|K|- 
ns)€ Rs 


vector of all traffic demands for all travel time intervals and all origin-destination pairs in 
the network. 

Let ? be the set of all simple paths (i.e., paths without cycles) in the network and let 
prs) C P be the set of all paths between an origin-destination pair (r,s). An element of 
P will be denoted by p, while an element of ? ^" will be denoted by p^. Each path is 
formed by one or more directed arcs.? 

The route flow departure ratet on path p € P during the k-th time interval will be denoted 
by fp ([veh/h]), the arc inflow rate on the link a during the k-th time interval will be denoted 
by q® ([veh/h]). 

The average route travel cost on the route p € ? when starting during the k-th time 


interval will be denoted by c®, the average link travel cost on a link a during the k-th time 


interval will be denoted by c® ({euro]). 

The route and link tolls, times, costs, and flows are related through a dynamic route-arc 
incidence indicator et (K) € {0,1}, which equals 1, if the travelers entering the route p € ? 
during the k-th time interval enter the arc a during the k’-th time interval, and 0 otherwise. 
We will assume that the route times, costs, and tolls are additive?, and that the following 
conservation constraints hold, i.e. Ó 


yr, DEN, e 


k'EK aca k'EK aca 
(k), (K) (K K k), (K 
P-y ya, Moy ye, (5.2) 
KEK aca keX acea 


'We do not consider the so-called departure time choice, as our main focus is on the optimal strategy for the 
road authority. This option is considered in, e.g. [48]. 

?Blastic demand road pricing models are introduced in, e.g., [84]. 

3Note that the order of links matters, the expression p = (6, 1,4) means that route p is formed by three links, 
where 6 is the first one, 1 is the second one, and 4 is the last one. 

“In the reminder of this chapter we will use the term “route flow” instead of the “route flow rate". 

5In reality, this does not need to be the case. For research dealing with non-additive costs, tolls, or flows we 
refer the reader to ied. 

Since some of the variables have to be rounded off, additional discussion about consistency of these equation 
is needed. Such a discussion can be found, in, i.e., Ea. 
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q 0949 — £ n (5.3) 


pee (rs) 


For each link a € A, the link travel cost ce for the k-th time interval is a linear combi- 
nation of the actual link travel time 1; and the actual link toll 6, with coefficients & [veh/h] 
and 1, i.e., 


c9 For ze, (5.4) 


where & [euro/time unit] is called the value of time, which is supposed to be independent of 
7 


q. 

Note that the link cost does not need to be strictly increasing with respect to the actual 
link flows, as the toll functions need not to be (strictly) increasing. It may seem counter- 
intuitive to have toll functions decreasing with the traffic flow, however, this phenomena 
was already encountered for the static optimal toll design problem in Chapter[4] In contrast, 
in, e.g., is. the link costs are assumed increasing with link flows. 

Let q®), t. and s® denote for the k-th time interval a vector of link flow rates on all 
links, a vector of link flows on all links, and a vector of link costs on all links, respectively, 
i.e., 


a i e 
gef? |, wel? |, jou | (5.5) 
q | A t b qa 


Let q, t, and ¢ denote the vectors of the link flows, the link travel times, and the link 
travel costs for all time intervals, 1.e., 


Po 10) cO 
Q) e (2) 

e q e t e S 

asl EI. [,s.m| | (5.6) 
qx lx) lll) 


Similarly, let us define 


k k k 
TEMPE. 
wer | wel? | wel 2 | 

2 i E 

[4 P P 

fo NU cO 
f def f 2) def 7) def co 
= n T= 5 c= 

plied (xl) Aa) 


"There are various ways in which the route cost functions can be defined, a common way is based on so-called 
generalized cost function, as mentioned in (45). 
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For each link from 7 and each time interval a traffic-flow dependent toll can be imposed. 
The traffic-flow dependent toll on link a € 7 will be denoted by e(? (-). Unless stated dif- 
ferently, this toll will be for each k-th time interval defined as a polynomial function of link 
flow for the same time interval and on the same link, i.e., 


M 
k k m),(k pn m),(k 0 for aea\T, 
e (a) = wi? (ai?) 5 a, for aET, (5.7) 


with M € No. By definition, w is constrained va” > 0 such that 


(k) f. .(K) =0 for aea\T, 
a (a ){ >0 for acq. (5.8) 


More advanced toll functions include traffic flows from previous time period, but we are 
looking for a very simple scheme improving the system performance, therefore we restrict 
ourself to toll functions in the form (5.7). Vectors 


0 g0) 
"THES " 92) 

(€ (lx) 

8410) 0 


are vectors of link toll functions during the k-th time interval and vectors of all link toll 
functions for all time periods, respectively. Coefficient vectors will be defined as follows: 


we i wb 
2, k 2) 

w w n 
M) (X) (Ix) 

wy Wia " 


Let wm € w=) min, wenns] for all m and let sets wi, W 9. and W be defined as 
follows: 


wi? 9€ foie, Damar) y. x [WO max] (5.10) 
ws def (w) [4| ow def (wo) |X| . (5.11) 


with we) min (m) max c R, (m) min. < yy(m).max for Vm c {1,...,M}. Clearly, w(? is a 
subset of RM and thus wi), wh), and W are convex and compact sets. It is assumed that 
w ew, wO ew, w € W forvk c x, ac a. 

Note that while coefficients we can be negative, the toll has to be nonnegative on all 
links, as stated in (5.8). 

With M = 0 in equation the toll level becomes time-varying, but not directly de- 


pendent on traffic flow (although this toll will be influenced by changes in the traffic flow 


76 5 Dynamic Optimal Toll Design 


pattern). In that situation the toll on the link a; will be set as wi € RO, and the vectors 
pu NH 
(0),k 2 
wih) der | 2 | a ue 
(0). (la) 
al = 


will be vectors of time-varying, but traffic-flow invariant tolls for the k-th time interval, and 
of time-varying, but traffic-flow invariant tolls for all time intervals, respectively. 


Let Wo be defined as set W with M = 0, i.e., Wo = win, (0) mas , with 0 < 


wO)min < yy(0).max T et w € Wo. Clearly Wo is a subset of RQ and a compact set. 

We also introduce the matrix M = {0,1}!%°!*/?|, which is the origin-destination pair- 
path incidence matrix. Its element in the (r‚s)-th row and p-th column is 1 if the route p 
starts from origin r and finishes in destination s and 0 otherwise. The traffic flow feasibility 
is described by 


Mf) =a, (5.12) 


5.2.1 Game-theoretic interpretation of the optimal toll design problem 


The problem of the dynamic optimal toll design can be seen as an inverse Stackelberg game. 
Two possible interpretations from the game theoretic point of view are possible: 


e The drivers, as followers, choose in each time period routes from their origins to 
their destinations so as to minimize their actual or perceived travel costs. Therefore, 
their decisions are their route choices. Because the average traffic flows are depen- 
dent upon these decisions and the road authority as the leader sets dynamic tolls as 
functions of the average traffic flows in the network, these tolls are also composed 
functions of the drivers' decisions. 


Because the travelers are uniform, all of them can be seen as one super-player, who is 
the follower in the one leader — one follower inverse Stackelberg game with the road 
authority as the leader. The decision of this super-player would establish the average 
traffic flows in the network. The dynamic tolls are the functions of the follower’s 
decisions in this game. 


5.3 Drivers’ behavior — dynamic traffic assignment 


This section formulates a macroscopic dynamic traffic assignment (DTA) model that de- 
scribes user-optimal flows over a network in which each driver chooses his/her preferred 
route from origin to destination, based on the time-varying conditions in the network. A 
driver starting his trip during the k-th time interval will influence the traffic conditions in 
this interval as well as the traffic conditions during later time intervals. The network con- 
ditions in the k-th time period depend on the conditions in previous time time periods. The 
travel behavior model used in this thesis can be found in, e.g., or [19]. 


5.3 Drivers’ behavior — dynamic traffic assignment TI 


The standard DTA models consist of a dynamic travel choice (DTC) model and a dy- 
namic network loading (DNL) model. 

The DTC contains a path choice model in which all travelers are distributed on all avail- 
able routes such that some kind of dynamic user equilibrium is achieved. Both deterministic 
and stochastic equilibria will be considered. 

In Section [5.3.1] the dynamic traffic equilibria used in this thesis are defined and dis- 
cussed. In Section[5.3.2]the dynamic network loading model will be formulated. 


5.3.1 Dynamic traffic equilibrium conditions 


In the problem of traffic assignment with given traffic demand, each user chooses a certain 
route from his/her origin to his/her destination. The rules according to which the users 
decide which route to use have to be specified. The behavioral model used in this thesis 
is the so-called Dynamic Traffic Equilibrium, as stated in, e.g., [19]. We consider both its 
deterministic and stochastic variants. 


Definition 5.1 (Dynamic deterministic traffic equilibrium) 

The traffic network is in the dynamic deterministic traffic equilibrium, if for each origin- 
destination pair, the route travel costs for all users traveling between a specific origin— 
destination pair and departing during the same time interval are equal, and lower than the 
route travel costs which would be experienced by a single user on any unused feasible route, 
i.e., if for all (ns) E RS, p € P (5) the following statement holds: 


if f()20, then c= min cO, vkex, peet”, (ns) ERS: 


pep o5 P 


if Dn —0, then c® > min cp’, vkex, perl), (ns)ems. 
pcP ns 


Definition 5.2 (Dynamic stochastic traffic equilibrium) 

The traffic network is in the dynamic stochastic traffic equilibrium, if for each origin- 
destination pair, the perceived route travel costs for all users traveling between a specific 
origin-destination pair and departing at the same time instant are equal, and lower than the 
route travel costs which would be experienced by a single user on any unused feasible route, 
i.e., if for all (r,s) € RS, p € P 5) the following statement holds: 


If D 20, then ep = min cj, Vkex, per), (ns) ERs; 
PEP! rs 


if fe —0, then ep > min čp’, VkKEK, pe prs), (ns) ERS, 
pers 


x(k) « f 
where e ) is the perceived travel cost on the route p. 


As in Chapter [5] Section [4.3] we assume that in equilibrium state, the so-called logit- 
based dynamic stochastic equilibrium takes place. This means that the following equation 
applies for each p € e ^9), k € K: 

k 
() exp(-uch) POKO 
D == a . 
Leer (r,s) exp(—u cp) 
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5.3.2 The dynamic network loading model 


The dynamic network loading (DNL) model is formulated as a system of equations ex- 
pressing link dynamics, flow conservation, flow propagation, and boundary constraints. The 
DNL model simulates the progression of the route flows on the network, yielding dynamic 
link flows, link volumes, and link travel times developing over time. The DNL model used 
in this thesis is adapted from and can be expressed by the following system of equa- 
tions: 


(k) 
kia 
D )- up (5.13) 
k) qi, if a isthe first link on path pe 2), 
Map 0 - mE (5.14) 
Vom Were a— is the preceding link of a. 
pee (rs) 
ve = Yo (5.16) 
peo (rs) 
K K 
“=, (« I J^ (5.17) 
Kk 


where i9 is an approximation of the link travel time. In addition, the link travel time 


function for the k-th time interval is a nondecreasing and link-specific function of the link 
volume on the same link for the k-th time interval. 

Equation (5.13) is a flow propagation equation. It describes the propagation of the 
inflows ult) through the link and therefore it determines the outflows 9. Additionally, it 
relates the inflows and outflows of link a at the k-th time interval of vehicles traveling on 


is T" (k) 
route p from origin r to destination s. The » ) is defined as follows: 


iO. ie 19 C 05, -0.5)A). (5.18) 


We do not assume explicitly that a FIFO (first-in first-out) condition has to be satisfied. 
Equation describes the flow conservation conditions. If link a is the first link on 
route p, the inflow rate is equal to the corresponding route flows determined by the route 
choice model. If link a is not the first link on the route, then the inflow rate u, is equal to 
the link outflow rate yt) p 
Equation states that the total link inflows are determined by adding all link inflows 
for all routes that flow into link a at that time interval. 


of the preceding link a—. 


Equation (5.16) states that the total link outflows are determined by adding all link 
outflows for all routes that flow out of link a at that time interval. 

Equation (5.17) defines the link volume x) i.e., the number of travelers present at the 
beginning of the k-th time interval on link a. 
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5.4 The problem formulation 


Similarly as in Section 4.5]the goal of the road authority is to choose an optimal w*, min- 
imizing his/her objective function. As described in Section the problem is an inverse 
Stackelberg game. The problem of the total travel time minimization can be formulated as 
follows: 


Find 
(PD)4 w* =argmin gw Z (q(w),w), 
subject to q € DUE (w), where 0 is defined by and (5.8). 


The expression q € DUE (w) reads as “link flow vector q is a result of a used dynamic 
user equilibrium (DUE) model when a polynomial toll function with coefficient vector w is 
used." . 

The "standard" Stackelberg problem would be defined as a subproblem of (PD): 


Find 
(PDo)4 wo = argminw ew, Z (a(wo), wo) 
subject to q € DUE (w). 


5.5 General problem properties 


Note that problem (PD) is a nonlinear programming problem, similarly as problem (PD) 
introduced in Section [4.4] Also, the problem (PD) has at least one solution if the DUE 
constraint represents a compact set of (w,q(w)). 

If for any given w the set DUE (w) is a singleton, w — q is a one-to-one mapping. In this 
case, the continuity of q with respect to w will guarantee that the constrained set of (PD) is 
closed, which implies the solution existence of (PD) since q and w are bounded. 

In general, DUE (w) may have multiple solutions in terms of q and thus DUE (w) may 
not be a singleton. In this case, DUE (w) is a point-to-set mapping of w Ba). The solution 
existence of (PD) will depend on the compactness of the graph DUE (w), defined as 


VY (w,q) = ((w,q)|g € DUE(w), Vw E W}. (5.19) 


Theorem 5.1 The problem (PD) has at least one solution if the following conditions are 
satisfied: 


i. The set DUE (w) is nonempty and compact for Vw € W, 


ii. Let w,w € W and let q € DUE(w), q € DUE(w). For each € > 0, there exists 6 0 
such that if |w — w|| < ò, then 


max min  ||g—4|| <e. 
Vq€DUE(W) VJEDUE (W) 


iii. The link travel cost functions on all links are continous functions of the link flows on 
the same links. 
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Proof: Let R(0,£) be an open ball with radius €. Then Y E DU E(w) 4- R(0,£) is an open 


set containing DU E(w). Let us define an other open set Z a iw: ||w-w|| < ò} containing 
w. According to condition ii. in Theorem[5.1] for any € > 0, there exists ö > 0 such that 
max min ||¢—q|| <e, 
VqeDUE(W) VqcDUE(W) 

which is equivalent to UwezDUE(w) C 9. Thus, under ii., the point-to-set mapping of 
DUE (w) is upper-semicontinuous. Together with condition i. it implies that the point-to-set 
mapping DUE (w) is closed on set W. Thus the graph Y (w, q) defined in is closed by 
Theorem|4.2] Also, under i., DUE (w) is bounded for any w € W. Since W is a bounded set, 
the graph ¥ (w,q) is bounded as well. Thus, graph w € W is compact. Together with iii. and 
the fact that W is compact, we can conclude that (PD) has at least one solution, since it is a 
nonlinear programming problem with a continuous objective function defined on a compact 
set. 


Theorem 5.2 Problem (PD) is strongly NP-hard. 


Proof: The proof follows from the fact that the problem (P) is a special case of the problem 
(PD) (with k = 1) and from Theorem 


5.6 Solution methods 


The methods used to solve the problem (PD) are those introduced in Section adjusted 
to the dynamic environment. The problems with the drivers driven by the dynamic deter- 
ministic user equilibrium can be solved analytically, as long as their scale is not too large. 
The problems with the drivers driven by the dynamic logit-based (stochastic) equilibrium 
will be solved by an algorithm containing a neural networks approach for solving the upper 
level of the problem and the method of the successive averages for the lower level of the 
problem. Since the dynamic deterministic user equilibrium is a special case of the dynamic 
logit-based (stochastic) equilibrium, also the deterministic problems can be solved using the 
neural-network based approach. 


5.7] Case studies 


In this section the solution methods introduced in Section and mentioned in Section[5.6] 
will be applied on a number of case studies. Problems introduced in Section are 
simplified versions of problems (PD) and (PDo), respectively, with travelers driven by the 
deterministic dynamic user equilibrium, applied on a two-link network. An analytical solu- 
tion is given. Problems introduced in Section[5.7 2]are simplified variants of problem (PD) 
and (PD), too, with the drivers driven by stochastic user equilibrium. A numerical solution 
is given. 


5.7.1 Three-links network 


In this section problems (PD) and (PDo) introduced in Section [5.4] played on the three- 
link (route) parallel road network depicted in Figure will be dealt with. Two alternative 
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3 - untolled 


Figure 5.1: One origin-destination pair network with 3 links. 


objective functions Zı = Zi (q(w),w) and Z2 = Z2 (q(w),w), defined as 


Zi (q(w),w) = af (w) tw), (5.20) 
Zo (q(w),w) € —q' (w) - 6(w), (5.21) 
will be considered.? 
We assume that K = {1,...,7}, A = 1 [h], d ^9 (0 = 2000 [veh], d”*”-@) = 2000 [veh], 
d (733) = 3000 [veh], 4^9 (4) = 3000 [veh], q/^95) = 2500 [veh], 4(^9.(9 = 2000 [veh], 


d (^90) = 2000 [veh], a = 8 [euro/h], 14) = Bagh +8, ae {1,2,3}, c Saus) +00". 
òi = 4, 82 = 3,65 = $, Bi = m, B2 = m: Ps = ze 


5.7.1.1 Total travel time minimization 


Let the road authority minimize the total travel time of the network. In the following four 
games we will consider different toll variants in Stackelberg (traffic-flow invariant) and 
inverse Stackelberg (traffic-flow dependent) setting to see how different classes of toll func- 
tions influence outcomes of the game. To be able to give an objective comparison we will 
focus on inverse Stackelberg strategies with toll functions having the same number of un- 
known parameters as Stackelberg strategies to which the inverse Stackelberg strategies are 
compared to. The goal is to find an inverse Stackelberg strategy that does not increase the 
problem complexity? and that provides better outcomes for the road authority. 
In Section[5.7.1]the best possible outcome for the road authority is discussed. 


Game 1 


Let only link 1 be tolled. Two problems will be compared: 


; TC ; > F def 
e A problem of total travel time minimization with uniform (constant) toll, i.e., e? = 


0; ce RO. 
e A problem of total travel time minimization with toll defined as a $j-multiple of an 


actual link traffic flow on link /j, i.e., o? (a) de, q”, & ec RO. 


5Other possible objective functions for the road authority can be, for example, unreliability of the network sl, 
negative of the surplus of the network [84]. etc. 
?This is important for possible real-time applications. 
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The optimal toll for the first problem is = = 0.39 [euro] and yields a total travel time 
of 9590.79 [h]. A slightly better outcome, 9583.12 [h], can be reached in the second game 
with an optimal value of &; equal to 5 = 0.36 - 10°. The outcomes of the games are 
very close to each other because the problem was solved with respect to only one parameter. 


Game 2 
Let link 1 and link 2 be tolled. Two problems will be compared: 


def 
e A problem of total travel time minimization, where toll is uniform, i.e. Ae ye 0,€ 


R? ‚ae (1,2). 


e A problem of total travel time minimization with toll on link a (a € {1, E defined as 
a €,-multiple of actual link traffic flow on link a, i.e., e(9 (di qa 2 et P ; ) €, ER? 
ae {1,2}. 


For the first problem the optimal tolls on links 1 and 2 are 5 [euro] and i [euro], respec- 


tively, and yield a total travel time of 9590.79 [h] (the same outcome as in the previous case). 
The optimal values of £j and & for the second problem are 0.50- 107? and 0.51- 102, 
respectively, and yield the outcome 9578.36 [h]. The traffic-flow dependent toll is accom- 
modating to the traffic conditions in a better way, therefore this toll yields better outcomes. 


Game 3 


Let only link 1 be tolled. Two problems to be compared are: 
e Find o? minimizing the total travel time of the system, where 


g(t) def 0, kc11,2,6,7), 
1 61, k € {3,4,5}. 


e Find e? (a) minimizing the total travel time of the system, where 


(k) f. (K)Y def Eiq™, k€ {1,2,6,7}, 
9j (di ) z (k) 
Eig) A ke 13,4,5}. 


The optimal values of 0; and 6; in the first game are Au = 0.39 [euro] and HR = 0.39 [euro], 
tespec Nel and yen the total travel time 9590.79 [h]. The optimal values of &| and & 
are 05 = 0.43. 10? and X5 = 0.44 - 1077, respectively, and yield the total travel time of 
9582.68 [h]. 


Game 4 


Let links 1 and 2 be tolled. Two problems to be solved are: 
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Table 5.1: The optimal link traffic flows [veh/h] and the link travel times [h] for the problem 
of total travel time minimization. 


e Find 99, o, minimizing the total travel time of the system, where 


(k) def ) Og € Ri 
0; = en _ ( 
0, ER, 


a € 


D, ke {1,2,6,7}, ae {1,2}, 


, k€(3,45), ae {1,2}. 


rcd 
= 


e Find e(9, el), minimizing the total travel time of the system, where 


oP (a) = tdP, ke {12,67}, a€ {1,2} 
! E, qi? k € {3,4,5}, ae {1,2}. 


The optimal values of 01, 02, 6;, and 65 for the first problem are 5 [euro], 5 [euro], 
1 


3 [euro], and i [euro], respectively, and yield a total travel time of 9649.51 [h]. The optimal 
values of £;, £2, £1, and & for the second problem are 0.77 - 10~2, 1.29- 1072, 0.78 10, 
and 1.26 - 107°, respectively, and yield a total travel time of 9577.38 [h]. 


General outcome 


Minimization of the total travel time function with respect to the traffic flows yields the link 
traffic flows and the link travel times as depicted in Table 

If these traffic flows and travel times are the travelers' response to the tolls, minimal 
total travel time D ~ 9577.29 [h] will be obtained. This means that the second strategy 
from Game 4 yields a total travel time close to the optimal outcome. The optimal outcome 
9577.29 [h] can be reached if more parameters in the toll functions are included. In Table[5.2] 


we find the optimal linear inverse Stackelberg strategy (o(9 E wP) q® + w) and the 


optimal Stackelberg strategy, minimizing the total travel time of the system. Since for the 
inverse Stackelberg strategy parameters wd are free (and, therefore, the solution of the 
inverse Stackelberg game is nonunique), clearly 7 parameters in the toll function are needed 
to obtain the optimal outcome. Obviously, by setting wi to 0 the optimal Stackelberg 
strategy will be reached. Therefore, it seems that with enough toll parameters the outcomes 


of the two strategies would be the same for general networks. However, it is difficult to 
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1 
2 
3 
4 
5 
6 
7 


bale aloe ales lee Tile zo zoe 


Table 5.2: The optimal link toll function coefficients for the inverse Stackelberg game (total 
travel time minimization). The optimal tolls for the Stackelberg game are e = 


8/15 [euro] and 0) = 8/15 [euro]. 


compute optimal time-varying toll on each link and for each time period in the real time, 
if the number of toll parameters is very high. That is why it is important to find strategy 
working better even with low number of toll parameters. 


5.7.1.2 Total toll revenue maximization 


Let us again consider the network depicted in Figure where the road authority maxi- 
mizes the total toll revenue of the system. The traffic flows imposing the best outcome for 
the road authority are unknown here, because the total toll revenue toll changes with change 
of toll strategy. 


Game 1 
We will first assume that only link 1 is tolled. Two problems will be compared: 


(k) def 
1 = 


e The problem of total toll maximization, where the toll is uniform, i.e., 0 01€ RO. 


e The problem of total toll maximization with toll defined as a €;-multiple of actual 
link traffic flow on link ly, i.e, 6]? (40) SE, qi, & e R9. 
The optimal toll for the first problem is oe = 2.48 [euro] and yields a total toll revenue 
of 9690.19 [euro]. The solution of the second problem is 6; = X and yields a total toll 
revenue of 9931.46 [euro]. 


Game 2 
Let link 1 and link 2 be tolled. We will compare two problems: 
(k) def 


e The problem of total toll maximization, where the toll is uniform, i.e., Oa ) de 0,€ 
R^, ac(12). 


5.7 Case studies 85 


e The problem of total toll maximization, with toll defined as a &,-multiple of actual 
link traffic flow on link a, i.e., 
k k)\ def k 
e (P) EE ad, Eerd, ae {1,2}. 


The optimal tolls for the first problem are Tos £z 4.30 [euro] and a = 4.10 [euro] for links 1 


and 2, respectively, yielding the total toll revenue 26071.23 [euro]. For the second problem 
the optimal values of 6, and & are 0.77 - 10^? and 1.27 - 1077, respectively. The resulting 
total toll revenue is 26794.74 [euro]. 


Game 3 


Let only link 1 be tolled. We will compare two problems: 
e Find e maximizing the total toll revenue of the system, where 


g(0 def f Or, & € {1,2,6,7} 
1 l6, ke {3,4,5}. 


e Find o? (a) maximizing the total toll revenue of the system, where 
Eig, ke {1,2,6,7 
o9 (a) def 5141 , €1 VION } 
DNA Eig), ke {3,4,5}. 


The optimal values of 0; and 6; for the first problem are = = 2.16 [euro] and = & 2.90 


[euro], respectively, and yield the total toll revenue 9901.83 [euro]. The optimal values of 


& and & for the second problem are both equal to as and yield a total toll revenue of 
9931.46 [euro]. 


Game 4 
Let both link 1 and 2 be tolled. We will compare two problems: 
e Find Ql") and of maximizing total toll revenue of the system, where 


e| GERD), ke{1,2,6,7}, ae {1,2}, 
a = zy (0 
Ri 


SER", ke {3,4,5}, ae {1,2}. 


e Find e? (a) ; e (a?) , maximizing the total toll revenue of the system, where 


a? (qo) & Eq, kE{1,2,6,7}, ae {1,2}, 
Bas) ke {3,4,5}, ae {1,2}. 
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AC 
wi 


T00 
pO ,,00:0),,(0.0) 


WOC DO, 
0.) 


2 
4000 (10,00). DD OM 
200, 00) 


Table 5.3: Optimal link flows: Total toll revenue maximization with linear toll functions. 


The optimal values of 04, 05, 61, and 6) for the first problem are +8 & 3.73 [euro], 2 N 


5.07 [euro], = & 3.53 [euro], and a ~ 4.87 [euro], respectively, and yield a total toll 
revenue of 26706.15 [euro]. The optimal values of &1, &2, & , and 2 for the second problem 
are 0.77 - 102, 1.29 - 102, 0.78 - 1077, and 1.26 - 107°, respectively, and yield a total toll 
revenue of 26795.01 [euro]. 


Since the total toll revenue function will vary depending on the chosen structure of the 
toll functions, it is impossible to get the maximal total toll revenue before knowing the 
toll structure used. In the following game the optimal value of the total toll revenue with 
linear tolls will be computed, as this toll brought the best possible outcome when various 
polynomial toll functions were tested. 


Game 5 


We will consider the situation, in which the road authority maximizes the total toll revenue 
of the system by setting tolls defined as follows: 


ef? (a) EDO OO, 0D (49) wg 1,99, 522) 


Provided that coefficients we j=1,2, kelIl,...,7} are negative, local maxima 
of the total toll revenue function with respect to the link traffic flows will be reached with 
flows depicted in Table[5.3] These traffic flows are dependent on wy) and we (j= 
1,2,3, k 2 1,7) 

The road authority maximizing the total toll revenue, aiming to influence the travelers 
such that the traffic flows depicted in Table [5.3] will be obtained, has to take into account 


the dynamic deterministic user equilibrium conditions. If all three links are used, these 
conditions will yield coefficients p and pl) as depicted in Table 


87 
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3-550w( O 675 w 195759, 00) yh O 


80w 9 (-73+17500w5 9) 
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-93150w 0 0.0) 
50004009 
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80w 9 (-73417500w5 9) 
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3-6715w ® 559,10.) 


2 


4315 (00 
-93750w6 O (0,6) 
43750) 

-93750 009 (0.0) 
1875.20) 


-93750. 09) (04 


40w$P 9 (2474993151 9) 
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3-675 wt!) sso yh) 4 


2 


-93750w{) 


200 (—247+09375 wi) 


3675 wi) sso whl) 4 


-93750w1 ^ 


Table 5.4: Coefficient of linear toll functions yielding local maximum of the total toll revenue 
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7.4795 7.0757 
7.4795 7.0757 
10.6860 10.2769 


10.6860 10.2768 
9.0533 8.6608 
7.4795 7.0757 
7.4795 7.0757 


Table 5.5: Optimal toll function coefficients and resulting tolls [euro]: Total toll revenue 
maximization. 


2 4 


1 1-tolled 3 4 5 


Figure 5.2: Chen network. 


Here we we k — 1,...,7, are free. However, after substituting WO), wo, 


k — 1,...,7, from Table[5.4]into the total toll revenue function and maximizing the obtained 


function with respect to we wh, k=1,...,7, the values of the coefficients of the toll 


function can be obtained. These coefficients are depicted in Table[5.5]and yield the maximal 


toll revenue 2.6795 - 10^ [euro]. Substituting the coefficients wi, XQ X0 


wi from Table [5.5]into will result in toll values gl”) and o9 as depicted in the 
same table. Obviously, toll defined as a polynomial function (of the actual link flow) of 
degree higher than 1 will not lead to a system performance improvement (with respect to 
the system performance when the classical Stackelberg strategy is employed), thus this is 
the best outcome achieved. This means that also the second strategy from Game 4 is the 
best strategy that we found. 


,and 


5.7.2 Chen network 


In this section case studies with the Chen network consisting of 6 links, 2 origin-destination 
pairs (1,5) and (3,5), and 6 routes will be investigated (depicted in Figure[5.2). The traffic 
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on each link flows from the node with lower identification number into the node with higher 
identification number. Only link 1 is tollable, the toll is defined as 


DO OD, 


with the traffic volume x®. ) Nonnegativity condition applies here, too. 

For each directed arc a € A the following parameters are initially given: link length 
Sa [km], maximum speed 87° [km/h], minimum speed gmin [km/h], critical speed port 
[km/h], jam density y [pcu/km], where pcu denotes passenger car units, and the unre- 
stricted link capacity C; [pcu/h]. Dynamic link travel time for an individual user entering 
link a during k-th time interval (k € K ) is defined as 


D ws (5.23) 


where the link speed oi) [km/h] can be computed using Smulders speed-density function 


(see (7311): 


om, DD, if JP < gett, 
. (k) _ (jam 1 : 
oP = 4 JE". (gett omin) IEEE s dt xg 624) 
Jerit m Jim 
gmin if JË > jer 


with critical density J“ [pcu/km] defined as JS“ = C, / 9c. 
The road authority minimizes the total travel time of the system, i.e., 


WELL Y sd. 


ke X (r,s)ERS pee (ns) 


We assume that the logit-based dynamic stochastic equilibrium applies for the drivers. 


Case Study 1 


Four time intervals are considered, i.e., K = {1,2,3,4}, A = I [h]. The link properties and 


the travel demands are depicted in Table The other parameters are set as: u = 0.2, 


€ = 0.05, a = 8 [euro/h], win = —10, min. 5 (max _ 10, Ome _ 5, 


am 
epee eee] 


| 46940] I | 469.0) | 2 | 40945) | 3 | 46949) |] 


ERE Ean ERN EM 
1000 1500 2000 1500 


Table 5.6: Link properties and travel demands in Case study 1. 
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The neural-networks based algorithm was applied, with 33620 training data, 13297 vali- 
dation data, and worst accepted validation error equal to 1.196. Sixteen processors were used 
to compute the problem in a parallel way, where both grid search and neurosimulation were 
parallelized. The neural ANN function that approximates the total travel time function in the 
“best way" (See Section[4.6.3]for explaining this best way.) is smooth, twice differentiable, 
with more than one local minimum (shown numerically) and one global minimum. The 
minimum 14173 [h] is found at WI, wd, wi?) wo, wid, we», WD. 
wl) = [-0.50, 0.20, —0.03, 1.19, 0, 0, —0.04, 3.96]. Note that for the first and forth 
interval the optimal toll is decreasing with the current traffic volume. This phenomenon 
appears when other links are congested than the tolled link and the aim is to attract the 
travelers to the tolled link. 

With no toll the total travel time reaches 19542 [h] the optimal time-varying (but traffic- 
flow independent) tolls are gi!) = 23 [euro], e(" — 6.6 [euro], o — 9.5 [euro], e(? =7.4 
[euro], and yield total travel time of 1.7844 - 10^ [h]. 

The computational time of the FAUN simulator was 10.23 hours, the computational time 
of the grid search was 35.21 hours. This time can be decreased by using more processors to 
solve the problem. 


Game 2 


In this case study the number of time intervals will be increased to 8, with travel demands 
depicted in Table[5.7] Also, there are no boundaries on parameters of linear toll functions 


| qe | 1 | 46940) | EUM | 40540 | | 469.46) | 5) dis | 40546 | | 46940 | | 40548 | 


t EN EUM EM EM EM EUM EM EUM EM 
3,5) 1000 2000 3000 4000 4000 3000 2000 1000 


Table 5.7: Travel demands in Case study 2. 


and only 14122 training data and 9301 validation data were used. The worst accepted 
validation quality was set to 1.1%. The best-trained neural network was minimized using 
Matlab again. The approximation function is again twice differentiable, with multiple local 


minima, and one global minimum 29149.00 at p XO wi), XQ whl?) 
OD DD OD DO , (04. 06) OO DO OO DD OD, 
wi 0.8] 0.02, 2.62, —0.04, 3.20, 0.4, —0.93, 0.01, —1.32, 0.01, 0.99, 0.05, 


0.40, 0, 0, 0.02, —0.24]. 

Optimal toll decreasing with the current traffic volume appears in the first time interval 
and in the second time interval. With no toll the total travel time reaches 39659.20 hours. 
The optimal time-varying (but traffic-flow independent) tolls yield a total travel time of 
34822.60 hours. 

The computational time of the FAUN simulator was 7.15 [h], the grid search took 26.11 
[h]. This time can be decreased by using more processors to solve the problem. From the 
tests made after the computation it follows that the obtained solution is very accurate in its 
neighborhood (with an error of 1%), although a lower number of training and validation 
data was used. 
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Discussion 


In both case studies the traffic-volume (and hence traffic-flow) dependent toll improved 
the system performance remarkably. Also, phenomena of the toll decreasing with traffic 
volume was observed. The natural explanation for this phenomena is that the traffic should 
be attracted to the tolled link. 

The grid search is very time consuming, although the network used is very small. The 
speed of the solution process can be increased by further parallelization of both phases of 
the solution process. 

Generally, the time-varying but traffic-flow invariant toll can never lead to a better out- 
come than the traffic-flow dependent toll. This follows from the fact that the dynamic op- 
timal toll design problem with traffic-flow invariant toll is a special case of (PD). See 
for further explanation. 


5.8 Conclusions and future research 


In this chapter we have introduced the concept of the dynamic optimal toll design prob- 
lem with second-best traffic-flow dependent toll. We discussed existence of solution of this 
problem as well as its difficulty and proposed a solution algorithm, based on the algorithm 
used for the solution of the static version of the problem. In the case studies we have shown 
both problems solved analytically and problems solved numerically using the proposed al- 
gorithm. 

All the topics mentioned in Section [4.8] extended to the dynamic environment, can be 
subjects of future research. Additionally, departure time choice of the travelers is a topic 
calling for the further investigation. In the departure time choice of the travelers was 
considered, without the traffic-flow dependent toll and finding the optimal solution of the 
problem. 


Chapter 6 


Electricity Market Problem 


In this chapter we propose a simple formalization of the electricity market problem. The 
model aims to see the differences between the prices in the perfectly competitive market, 
in the market with one leader, and in the market with two leaders, playing Nash among 
themselves. 


6.1 Introduction 


The European electricity market is currently being transformed from a market with monop- 
olistic, national, and state-owned producers to a market with competing, private, and often 
multinational firms. This transformation is called liberalization. The speed and current state 
of this process vary among different European countries, from a near monopoly in the Czech 
Republic to highly competitive markets in Norway, Sweden, and the Netherlands [54]. 

Main aims of the liberalization are to bring benefit to consumers by lowering electricity 
prices and to cause more cost efficient electricity production. Little is known about the envi- 
ronmental consequences of liberalization. On the one hand, more cost efficient production 
may be beneficial for the environment, while, on the other hand, lower market prices imply 
higher electricity demand that may increase the burden on the environment. Moreover, in 
a highly competitive market an incentive to produce electricity with cheap, but often not 
environmental-friendly means, is increased. It is also assumed that quite recent develop- 
ments, such as the implementation of the European union's CO» emission trading system in 
2005, may have major environmental impacts. 

In the liberalization process, with various competing firms, the market and its rules 
are no longer fixed. The effects of liberalization on market structure can be illustrated by 
the recent development in Germany. Following liberalization, the initial 30 relatively small 
electricity producers were merged into four large producers in only few years of time. These 
firms have market power on the German market, but they also face competition from neigh- 
boring countries. The extent of international competition is limited by the transmission 
capacities between countries, but it is also affected by the market structure in these neigh- 
boring countries. Also, the character of electricity markets in countries that are neighbors 
of Germany will be changed by trade with Germany. 
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To get more insights into how the liberalization can influence the European electricity 
market, we will view the electricity producers in eight European countries: Belgium, Den- 
mark, Finland, France, Germany, The Netherlands, Norway, and Sweden as players in a 
game. We chose these countries because of the following reasons: 


e There exists a model including these countries (in isd) with different ways to model 
the firms’ behavior, thus we can compare our results with already existing results. 


e Real data about electricity prices, emission factors, and electricity productions are 
available 132]. 


In the game that we consider the electricity producers choose technologies for electricity 
production as well as amounts of electricity to be produced for different load periods. Only 
the game among electricity producers of different countries is played, the consumers’ de- 
mand of electricity is exogenous. This approach is reasonable in the situation, in which the 
selling price of electricity in each country has to be the same for each producer, i.e., the 
consumers cannot choose “cheaper” electricity from different producers. 

The number of producers per country is given, as well as parameters like electricity 
production costs and electricity production capacities, and the emission factors per country 
and per technology. These initial data are derived from real data and were taken from 
existing literature and electronic sources (32,|56). Additionally, shadow prices on emissions 
per energy producer can be set. 

Different game theoretic scenarios of firms behavior will be formulated, namely a per- 
fect competition, a Stackelberg game, in which in each country one firm acts as a first- 
moving Stackelberg market leader during dispatch, and a Stackelberg game with two leaders 
per country acting as first-moving Stackelberg market leaders, playing Nash among them- 
selves. Extension of the model to the dynamic game, in which investments can take place 
over a longer time horizon is formulated and explored as well. In all considered scenarios 
the possibility of electricity transmission between neighboring countries will be considered. 

Extensive studies of static energy models have been carried out. In 62] the Belgian, 
Dutch, French, and German electricity market were considered and the effect of market 
power among three static models was compared. One of these model, the nodal pricing 
static equilibrium model COMPETES, was additionally studied in [42]. In Pd and 
[49] the consequences of market power in the Nordic electricity market are considered. In 
11] and a static game theoretic model of the European gas market is presented. In 
[20] emission permit trading to a nodal pricing model to study strategic effects of holding 
NOx permits is added. In the electricity market with eight European countries was 
considered. However, the decision variable of the individual electricity producers was the 
so-called market power mark-up, unlike the quantities of electricity to be produced, as it is 
in our research. 

The contributions of the research presented in this chapter can be listed as follows: 


e A new game theoretical model of electricity market of eight European countries is 
developed. Our approach differs from those presented in the existing literature, in 
which other types of markets, electricity market with less countries included in the 
model, or different decision variables for the electricity producers are considered. 
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e Various scenarios, like a perfect competition case or a Stackelberg game with one 
leading producer and the rest of the producers being perfectly competitive, are in- 
cluded in the model. 


e Most of the input data for our model come from real measurements, presented in 
existing literature. Therefore, the improved version of the model can help to explain 
some recent changes in the real European electricity market. 


e Extension of the existing model to a dynamic model with the possibility of invest- 
ments is proposed. Such a dynamic model is applicable for real data to analyze the 
current and future phenomena in the European electricity market. 


This chapter is organized as follows. In Section|6.2Ja static model of electricity produc- 
ers in eight European countries will be formulated. In Section various case studies are 
performed. In Section an extension of the static model into a dynamic model with the 
possibility of investments will be introduced. The outcomes of the case studies, their rela- 
tion to the situation in the current European electricity market, and possible future research 
directions are discussed in Section[6.5] Our research helps to understand such a complicated 
process, as the liberalization of the electricity market is. 


6.2 Games of the European electricity market 


Let us consider a game with electricity producers (also called firms or companies) in differ- 
ent countries (also called regions) as players. The following countries are considered in the 
model: Belgium (BEL), Denmark (DEN), Finland (FIN), France (FRA), Germany (GER), 
The Netherlands (NLD), Norway (NOR), and Sweden (SWE). The number of producers 
for each country is given. Firms generate electricity through different technologies; each 
producer chooses technologies among technologies available to him/her and the amounts of 
electricity produced by the chosen technologies. A producer can own several power plants 
of different types, of which the total capacity for each, as well as variable production costs, 
is given. Producers maximize their payoffs by choosing the amount of electricity to produce 
with various technologies for various load modes!. Producer's payoffs consist of the income 
from sales of electricity in regional markets minus the variable costs of production. There 
are limitations in transportation capacity of electricity and moreover, production capacity 
of electricity is fixed on the short term. The electricity demand curve for each country is 
exogenous and comes from real measurement 1561. Trade is only feasible with neighboring 
countries and includes netting, which means that bi-directional flows between two coun- 
tries are permitted, as long as trade constraints are not violated. Emissions are assigned to 
producers based on the actual technology used. These emissions can be restricted, too. 

Let F and R be a set of firms and a set of countries included in the model, respectively. 
Let F, € F bea set of firms located in region r, and let |F;| be their number. Let / be a set of 
possible technologies for electricity generation, i.e., 


pem [wind energy, nuclear energy, ...), 


'The set of load modes in our research contains two possible loads: base load and peak load. 
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let J, C I be a set of technologies that are available in region r € R, and let Ip be a set of 
technologies available to firm f. Let L be a set of possible load modes, e.g., base load or 


peak load. Let K be an emission type set, i.e., K EI (acid, smog, ...]. 
Firm f € F, maximizes its surplus 7; [euro] defined as 


lIEL TER leL  r'eR Niel 


3 S Y Y, (easyer) Nh Y, (Esas) , oy) 


where sf; [GW] denotes the supply of electricity of load mode / from firm f into region 
r and p, ; [euro] denotes the market electricity price for region r’ and load mode I. Let h; 
[h] denote the number of hours of a particular load / per year, c; [euro/MWh] represents 
the variable production costs with technology i in region r, in which firm f is located, while 
di.f., 1 [GW] is the production of firm f with technology i for region r’ for load mode /. The 
supply of electricity of load mode / per firm f to region r’ denoted by sp» can be defined 
as 


def 
Sprig = (1— Ne) Waren (6.2) 
iel 


where A, € [0, 1] is the loss of electricity due to transport to region r’, initially given. More- 
over, the electricity supply is additive, i.e., the total supply of electricity S, ; [GW] for load 
mode / per region r’ can be computed as 


Sy = Y Sf) 


JEF» 


where S, ; [GW] is a total electricity supply to region r’ during load mode I. 

We assume that the price p,; is dependent on the total supply of electricity S,; and 
follows a constant elasticity of distribution (CED) (2), la with elasticity —£,; (€, > 0) de- 
pendent on region r and load mode /, the reference demand for electricity d? ı [GW], and the 


reference price of electricity p? ‚ [euro], and can be computed using the following equation: 


—£rl 
Dr, 
di (z) = » Spr — Sy. (6.3) 


rl fer 
The firm's regional market share Tf; € [0,1] can be then computed as 


Sf.rl 


— — (6.4) 
Y rer, Spa 


Tp rl 
Firms can trade electricity with other countries as well. The amount of electricity traded 
x», [GW] is defined as the exported amount of electricity from region r to region r’ minus 
the imported amount of electricity entering region r from region r’ (f € F,), i.e., 


Xp = Y Maire y NAM (6.5) 


fer, iel f'eF, iel 


There is a restriction on the maximum production capacity of each firm f, the technology 
i, and the load mode /. The maximum production capacity is complementary to the shadow 
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price u; fı [euro/MWh], which has a nonnegative value if the production with technology i, 
by firm f during load mode /, is equal to the maximum available production capacity: 


Hifi p dif ar) 20, wigs >0. (6.6) 
VER 


The amount of electricity traded is also restricted. It is is complementary to the shadow 
price T,» ;. This shadow price obtains a nonnegative value, when the trade restriction is 
binding: 


Trl (er, am) > 0, Try > 0, (6.7) 


with the maximum amount of electricity traded between regions r and r’ denoted by x 
[GW]. 

Emissions can also be limited. Due to the Kyoto protocol, firms have to reduce the 
amount of emissions, where the shadow price of emission constraint x* [euro/MWh] is 
nonzero as soon as the current amount of emissions is equal to a permissible emission 
ceiling E* [g], 


K » hiY Y Y stair. zd 20, K >0. (6.8) 


lEL r'eRicIfeF 


Emission factors or. [g/MWh] are associated with the region, in which firm f produces 
electricity. 

If we include constraints (6.6), (6.7), and into the problem of maximizing the 
surplus function (6.1), firm f maximizes Ly defined as 


Lr = Y Y Woes (l- Adair earlier) 


IcLr'eRicl 
=} uY misal X asia 
leL iel r'ER 
-Lh Y Teil à Lapres à Daran 
IEL ER f'cF,iel fer, iel | 
r £r 


-yc z uY Y Y oar -z) s (6.9) 


keK lEL  r'eRicl fler 


Karush-Kuhn-Tucker (KKT) conditions for maximizing the objective function can be de- 
rived from equation by taking the derivative with respect to the production of electric- 
ity qi p, for firm f € F: 


kcK £p 1 
0 € qi.p (6.10) 


Tg p 
0 E gips (s Figs yp d ioi — 0 An) pr a [ = Set) 
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Market share Tr; is defined by equation (6.4). Inequalities can be interpreted as 
follows: As long as the marginal revenues from electricity sales are not lower than the 
marginal costs of production, a power company is willing to produce electricity. 

Marginal costs can be derived from equation as follows: 


Che = Cir tuigt ttt Y, x ot (6.11) 
kcK 


The four components of the marginal costs can be interpreted as follows. The first term are 
the costs of the producing electricity. The second and third term are the scarcity price of 
maximum production capacity per technology and the transmission price related to trade, 
respectively. The fourth term represents the addition to the production costs due to an 
emission constraint. 

We substitute the marginal costs defined in equation (6.11) into equation to obtain 
the following necessary condition for firm f to produce electricity: 


Tp pl 
if! Co m (1 — A) pra [ _ = 


D) 20, qiprı>0. (612) 


rl 


6.2.1 Game formulations 


We will consider three possible games among the electricity producers: a perfect compe- 
tition (PC), a Stackelberg game with one leader per country (SG), in which the rest of the 
producers is perfectly competitive, and a Stackelberg game with two leaders per country 
(competitive among themselves), where the rest of the producers are perfectly competitive, 
too (NSG). We will denote an optimal quantity produced by firm f € F, for region r’, load 
mode /, and technology i € J as follows: 


e qf m , for perfect competition (PC); 


e qf = , for Stackelberg game with one leader per region (SG); 


e ql Ae ‚for Stackelberg game with two leaders playing (Nash game among themselves) 


per region (NSG). 


Perfect competition 


We consider a situation with uniform players having the same technologies available and 
the same restrictions on capacities. In a perfectly competitive market the companies enter 
the game if their utility from the game is nonnegative, i.e., the problem of producer f in 
region r can be written as: 


Find 
f.P * qmax 
(at i) € (0, FET) 
(PC) such that 


Lay (prispe) - Dae (Zel 2) =0. 
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Since the solution of (PC) might be nonunique, we will additionally assume that 


S YT : 
(a En) =arg min pry (), 


d; pu t 


i.e., in a perfectly competitive market the firms choose the quantities to be produced in order 
to minimize the selling costs, while their utilities are nonnegative. 


Stackelberg game with one leader per region 


We assume that there is one leading firm in each region acting as the first player, choosing 
(dl om ‘J so as to maximize Ly defined in (6.9), whereas other producers, which 
ILT Lier MER ICL 


are the followers, are perfectly competitive. The leader can in advance determine how the 
other producers will react to his/her decision and with this information the leader can choose 


his/her optimal (af = j 1) . For each region, the means of electricity production 
br d jet, ERIEL 
are available to the leader of this game only, and are initially given. 


Stackelberg game with two leaders per region 


We assume that there are two leading firms f, f’ in each region r acting first, playing Nash 
FNS 


qi prn so as to 


. N. 
game between each other and choosing (al f 5 i) ; ( ) 
WIE) iel ER LEL iel, "ER IEL 


minimize their profits Ly and Ly. Other producers, which are perfectly competitive, choose 
their production amounts per load and technology after the leaders have made their choice. 

In each of the three games we are interested not only in the payoffs for individual play- 
ers, but also in how their behavior influences emission levels, what technologies to produce 
electricity would be preferred, and what amounts of electricity will be traded among neigh- 
boring countries. 

Data used for computations are consumers’ demand of electricity per region, supply 
data (generation capacity and cost), trade data (interconnection capacity), distribution losses 
data, emission factors. These data are taken from 83], and are presented and dis- 
cussed below. 


6.2.2 Model specifications 
Demand and supply side of the model 


Within the electricity markets of the considered countries we distinguish 34 different elec- 
tricity producers or firms, as presented, together with net losses A, and values for reference 
demands d, in Table 

The demand side of the model consists of one sector per national market. However, 
there are different markets for peak load (high demand) and base load (moderate demand). 
We consider two load periods, namely a peak period (2096 of the year) and a base load (8096 
of the year). We additionally assume that demand at peak hours requires 90 96 of the total 
available capacity: 

dian = 0.9 F qux. 


iel 
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BEL DEN FIN FRA GER NLD NOR SWE 
no. of firms 2 3 3 2 5 5 7 7 
net losses 45% 65% 3.5% 68% 4.7% 3.9% 8.9% 8.2% 
avg. demand (GW) 9.04 3.75 8.72 46.88 54.45 11.48 12.66 15.46 


Table 6.1: Characteristics of eight European electricity markets. 


BEL DEN FIN FRA GER NLD NOR SWE 


BEL 2500 1400 

DEN 1750 950 1900 
FIN 70 1450 
FRA 2850 1150 

GER 1350 1750 3300 

NLD 1400 3300 

NOR 950 70 3035 
SWE 1840 2050 550 3035 


Table 6.2: Transmission capacities between the countries (MW). 


The price elasticity of the demand is assumed to be set to —0.4, as in 156]. The intuition 
behind this relatively high price elasticity is that it reflects the alternatives for consumers to 
choose their electricity supplier [83]. 

The model encompasses 12 different production technologies, which can be listed as 
follows: 


e conventional thermal power technologies: nuclear (N), coal (C), gas (G), lignite (L), 
oil (O). 


e combined heat and power production (CHP) technologies: gas (CHP-G), coal (CHP- 
C), oil (CHP-O), biomass (CHP-B), and other fuels (CHP-X). 


e renewable technologies: hydro (H) and wind power (W). 


Due to varying fuel and production taxes across countries the variable production costs 
differ across regions and technologies, but not across producers within each country. A 
summary of total production capacities in the countries included in the model is given in 
Table 


Trade and distribution losses 


Firms in our model are assigned to a specific country. Hence, no crossborder ownership is 
permitted. There is an opportunity to trade electricity among countries, with the following 
restrictions: 


e Trade via imports and exports to countries outside the considered countries is ignored. 


e A firmcan only trade with neighboring countries. 
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nuclear 
coal 
lignite 
gas 

oil 
CHP-gas 
CHP-coal 
CHP-oil 
CHP-bio 
CHP-other 
hydro 
wind 

total 


nuclear 
coal 
lignite 
gas 

oil 
CHP-gas 
CHP-coal 
CHP-oil 
CHP-bio 
CHP-other 
hydro 
wind 


coal 
lignite 

gas 

oil 
CHP-gas 
CHP-coal 
CHP-oil 
CHP-bio 
CHP-other 


BEL 
5.71 
2.95 


3.50 
1.20 
0.58 


0.10 
0.29 


1.40 
0.01 
15.74 


BEL 
6.14 
16.94 


24.22 
36.42 
13.29 


19.58 
19.94 


0.00 
0.00 


BEL 
920.0 


388.0 
877.3 
330.6 


503.4 
0.0 


DEN FIN 
2.64 
5.10 2.29 
0.04 0.90 
0.79 1.24 
2.58 1.80 
1.13 1.47 
0.16 
0.23 1.04 
1.44 
0.01 2.88 
2.42 0.04 
12.30 15.89 


DEN FIN 
6.14 
13.83 13.97 
23.81 
35.21 
13.08 
7.57 
19.58 
19.94 


20.28 
35.21 
11.21 
7.63 
19.58 
19.94 
14.59 
0.00 
0.00 


0.00 
0.00 


DEN 
972.2 


FIN 
915.9 


327.2 
692.6 
673.9 
948.9 


348.9 
877.3 
528.3 
776.1 
503.4 
819 2.1 


FRA 
63.18 
12.69 


1.89 
12.23 


6.64 
25.60 
0.08 


GER 
21.37 
17.86 
18.97 
13.82 
8.11 
0.99 
6.96 
0.30 


11.61 
0.36 


122.31 100.33 


FRA 
6.14 
15.19 


23.83 
38.84 


16.69 
5.84 
0.00 


FRA 
915.9 


401.9 
756.8 


1296.1 401.6 


GER 
6.14 
14.42 
15.50 
29.04 
38.70 
15.85 
7.84 
21.43 


0.00 
0.00 


GER 
970.0 
1219.7 
348.9 
877.3 
327.1 
33.1 
503.4 


NLD NOR 
0.45 
4.05 
7.17 
0.99 
4.66 
0.64 

0.20 
0.04 27.46 
0.44 0.01 
18.44 27.67 


Table 6.3: Electricity production capacities in 2000 (GW). 


NLD NOR 
6.14 
16.83 


23.25 
41.21 
12.78 


19.94 

16.69 
0.00 
0.00 


0.00 
0.00 


Table 6.4: Variable cost (euro/MWh) per technology in 2000. 


NLD NOR 
915.9 


411.0 
877.3 
327.1 


0.0 
403.4 


SWE 
9.46 


4.64 
0.13 
0.56 
0.65 
0.46 
1.00 
16.33 
0.25 
33.48 


SWE 
7.50 


39.83 
13.52 
11.73 
21.58 
19.94 
16.69 
1.18 

0.00 


SWE 


877.3 
327.1 
733.1 
503.4 
0.0 

403.4 
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Table 6.5: Greenhouse gas emission factors (kg CO» equivalents/MWh) per technology. 
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BEL 
coal 31.594 
lignite 
gas 5.901 
oil 21.821 
CHP-gas 2.174 
CHP-coal 
CHP-oil 2.486 
CHP-bio 7.160 
CHP-other 


DEN 
20.699 


2.174 
2.486 
19.833 
20.217 


31.692 


FIN FRA 
23.310 31.549 


4.522 15.435 
21.821 25.610 
6.848 

32.459 

2.486 

46.726 

83.071 15.435 
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GER 
23.307 
33.896 
4.522 
21.821 
2.174 
2.649 
2.486 


NLD NOR SWE 


28.365 

6.783 

21.821 21.821 

2.174 2.174 
2.649 
2.486 

7.160 12.288 

3.736 3.736 


Table 6.6: Emission factors for acidifying emissions (g acid equivalent/MWh) per technol- 


ogy. 
BEL 
coal 80.0 
lignite 
gas 0.0 
oil 21.0 
CHP-gas 0.0 
CHP-coal 
CHP-oil 1.0 
CHP-bio 30.0 
CHP-other 


DEN 
57.0 


0.0 
1.0 
0.0 
57.0 


0.0 


FIN FRA 
172.9 170.0 
0.0 0.0 
3.0 130.0 
0.0 

150.0 

2.0 

21.0 

195.0 0.0 


GER 
66.0 
96.0 
0.0 
2.0 
0.0 
10.0 
2.0 


NLD NOR 
17.0 


0.0 


2.0 
0.0 


1.0 


SWE 


21.0 
0.0 
10.0 
2.0 
233.0 
1.0 


Table 6.7: Emission factors for smog formation (g fine particles/MWh) per technology. 
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The transmission capacity within a country is unrestricted. The interconnection capacity 
among countries of the electricity network is restricted and the data is derived from ETSO 
data 

Electricity distribution after transmission through the electricity grid is accompanied 
by losses. These distribution losses are generally much larger than the transmission losses, 
which are ignored in the model. The distribution losses A, (r € R) occur when the electricity 
is distributed within a country, and differ across countries, as depicted in Table 


Emission factors 


Three environmental effects can be taken into account in the game: greenhouse gas emis- 
sions, acidification, and smog formation due to emissions of fine particles. 

Information about emission factors for all technologies per country is taken from 
and [56] and is depicted in Table Table [6.6] and Table For all technologies, the 
specific emissions of the 8 considered countries due to the electricity generation were deter- 
mined. Emissions due to construction and deconstruction of power plants, mining, extrac- 
tion, and transportation have been disregarded, as these emissions, including emissions of 
extraction and transportation, are rather small, and in the same range of those for wind or 
hydroelectric power. Consequently emissions of hydroelectric, nuclear, and wind power are 
set to zero, CO» emissions of biomass power are also set to zero. 


6.3 Case studies 


For each situation (PC), (SG), (NSG) the following three scenarios will be considered. 


e There is only one country in the model (The Netherlands); electricity transmissions 
with other countries are not considered. 


e There are two countries in the model (The Netherlands and Belgium), electricity 
transmissions between these two countries can be considered; transmissions with 
other countries are not considered. 


e All eight countries are included in the model; the electricity transmissions among 
these 8 countries can be considered. 


In Table [6.8] and Table [6.9] schemes of the case studies are depicted. The first tabular 
refers to the games without emission constraints. The second tabular refers to the problems 
with emission constraints consideration. We set the permissible emission ceiling E^ [g] is 
set to 50% of the average of the resulting emissions of the eight countries when game E1.8 
is played. 

In the Stackelberg game we will assume that the leaders have access to the means of 
electricity production depicted in Table 

For games with 2 and more countries there will be variant (c), denoting that the cross- 
border transactions are considered. 


6.3.1 Games with one country 


The problems involving only The Netherlands will be solved analytically. 
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Without.. Ge CPC 2 NSG 
emissions 


1 country El. El.2 E13 
2 countries E14 E15 E1.6 
8 countries E17 E18 E1.9 


Table 6.8: Scheme of case studies with no emission restrictions. 


wi SG PC NSG 
emissions 


] country E21 E22 E23 
2 countries E24 E25 E26 
8 countries E2.7 E28 E229 


Table 6.9: Scheme with case studies with emission restrictions. 


Game SG NSG 

BEL wind, hydro, nuclear wind, hydro, nuclear, CHP-gas 

DEN wind, hydro wind, hydro, CHP-coal, CHP-gas 

FIN wind, hydro, nuclear wind, hydro, nuclear, CHP-gas 

FRA wind, hydro wind, hydro 

GER wind, hydro wind, hydro 

NLD wind, hydro, nuclear, CHP- wind, hydro, nuclear, CHP-gas, coal 
gas 

NOR wind, hydro wind, hydro 

SWE wind, hydro wind, hydro 


Table 6.10: The available means of electricity production for leaders in Stackelberg game. 
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Games E1.2 and E2.2 


Maximization of the utility functions with respect to the quantities produced gives the fol- 
lowing outcome. With perfect competition (Game E1.2) and with all producers having 
equal access to the means of electricity production, the selling price of electricity is 17.23 
[euro/ MWh]. 

When the emission constraints are considered (Game E2.2), the selling price of electric- 
ity is 19.13 [euro/MWh]. 


Games E1.1 and E2.1 


Let the leading producer have access to the means of production depicted in Table|6.10]as the 
only producer. In game E1.1 maximization of his/her profit with respect to the constraint of 
nonnegative profit for other producers leads to a selling cost of 25.98 [euro/MWh], yielding 
him/her a profit of 55182.92 [euro], while the utility of all other producers is zero. 

When the emission constraints are considered (Game E2.1), the selling price of elec- 
tricity is 30.10 [euro/MWh] and the profit for the leader is 49819.10 [euro], while the other 
producers obtain a zero profit. 


Games E1.3 and E2.3 


Let the two leading producers as only producers have (symmetric) access to the means 
of production depicted in Table|6.10| Then maximization of their profit with respect to 
the constraint of nonnegative profit for other producers leads to a selling cost of 20.31 
[euro/MWh] and an average profit of 44632.41 [euro], while all other producers have a zero 
profit. 

With emission constraints included the selling cost is 26.15 [euro/MWh]. This cost 
yields profit of 41023.24 [euro] for each of the leading producers, while all other producers 
have a zero profit. 


Discussion 


In the case studies with only one country (The Netherlands), the selling price is remarkably 
higher in the Stackelberg games than in the perfectly competitive market, especially when 
the leading producers have access to more ecological means for electricity production than 
the rest of the producers. While the resulting prices of our case studies are still about 25 96 
smaller when compared to the actual situation in the electricity market, the influence of the 
type of competition on the electricity prices matches the trends presented in very well. 


6.3.2 Games with two countries 
Games E1.4 and E2.4 


If game E1.4 is played, the electricity price in The Netherlands is 19.42 [euro/MWh] and 
yields a profit of 50244.12 [euro] for the leader; the electricity price in Belgium is 17.99 
[euro/MWh]; the profit of the leading producer will be 61213.24 [euro]. 

If game E1.4(c) is played, the electricity price in The Netherlands is 18.35 [euro/MWh] 
and yields a profit of 46001.21 [euro] for the leader; the electricity price in Belgium is 15.85 
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[euro/MWh]; the profit of the leading producer is 57192.91 [euro]. Belgian firms will sell 
1400 [MW] to the Netherlands. 

If game E2.4 is played, the electricity price in The Netherlands is 22.32 [euro/MWh] and 
yields a profit of 44115.23 [euro] for the leader; the electricity price in Belgium is 18.56 
[euro/MWh]; the profit of the leading producer is 57234.11 [euro]. 

If game E2.4(c) is played, the electricity price in the Netherlands will be 20.15 [euro/ 
MWh] and yields a profit of 37125.24 [euro] for the leader; the electricity price in Belgium 
is 17.12 [euro/MWh]; the profit of the leading producer is 50259.44 [euro]. 


Game E1.5 and Game E2.5 


If both Belgium and The Netherlands are considered in the perfect competition case (Game 
E1.5), 10.41 [euro/MWh] and 18.12 [euro/MWh] are the selling prices in Belgium and The 
Netherlands, respectively. 

If emission restrictions are included, the prices are 12.99 [euro/MWh] and 19.99 [euro/ 
MWh], respectively. 


Game E1.6 and Game E2.6 


If both Belgium and The Netherlands have two leading producers, playing Nash among 
themselves,” and cross-border electricity transmissions are prohibited (Game E1.6), the 
game does not have a solution. The two electricity producers in Belgium cannot cover 
the demand on electricity. Together they can produce only 7.70 [GW] of electricity, while 
the initial electricity demand in Belgium is 9.04. If the demand would not need to be sat- 
isfied, the optimal strategy for the identical leaders would be to set the price of electricity 
infinitely high. 

If Game E1.6(c) is played, the situation is solvable. Moreover, the electricity producers 
in Belgium cannot set the electricity prices arbitrary high, as they are limited by the elec- 
tricity prices in The Netherlands. A solution to the problem is as follows: The electricity 
price in both Belgium and The Netherlands is 18.25 [euro], the average profit of the Dutch 
producers is 73140.23 [euro], the average profit of Belgian producers is 23095.18 [euro]. 
If Game E2.6(c) is played, the selling price of electricity for both Belgium and the Nether- 
lands will be 19.31 [euro] and the average profits for the Dutch and Belgian producers will 
be 65232.13 [euro] and 18123 [euro], respectively. 


Discussion 


As in the game with one country, the perfect competitive market yields much lower elec- 
tricity prices. The prices will be remarkably increased if emission restrictions are included. 


6.3.3 Games with eight countries 


Since the analytical computation of the solution of the problem with eight countries would 
be extremely time-consuming, the problem was implemented and solved numerically in 
Matlab. 


?In Belgium these two producers are only players. 
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Game El7 El7(c EIS  EL8(c) EL9 EL) 
BEL 20.73 1841 10.04 823 1598 1321 
DEN 20.03 19.97 598 544 14.98 14.72 
FIN 2032 2028 7.81 523 15.05 14.88 
FRA 2035 1944 820 6.87 17.21 16.88 
GER 18.95 1795 7.87 6.62 11.22 10.99 
NLD 2113 1925 1585 12.00 19.54 1822 
NOR 13.21 12.59 001 0.01 925 9.01 
SWE 1732 1545 137 1.08 14.21 13.54 


Table 6.11: Resulting selling costs for base load period for games with eight countries. 


Game E1.7(e) EL8(c) EI.) 
BEL-FRA 1320/1500 0/2850 1440/1410 
BEL-NLD 890/50 1400/0 1000/25 


DEN-GER 1460/1300 1750/0 1500/750 
DEN-NOR 60/800 0/950 — 50/900 
DEN-SWE 210/800 0/950 100/880 
FIN-NOR 20/65 0/70 10/68 
FIN-SWE 320/1800 0/2050 200/1900 
FRA-GER 850/275 1150/0 910/105 
GER-NLD 2950/1500 3300/0 3005/545 
GER-SWE 200/455 0/550 150/505 
NOR-SWE 1420/2650 0/3035 720/2810 


Table 6.12: Electricity traded between neighboring countries [MW]. 


For each of the three games we will consider both variants with and without electricity 
transmissions between neighboring countries. 

The variants of the games, in which cross-border electricity transmissions are allowed, 
will be denoted by (c). The resulting prices for the base load period are mentioned in 
Table |6.11} whereas the amounts of electricity traded between the neighboring countries 
are depicted in Table In this table, 1320/1500 in column BEL-FRA illustrates that 
1320 [MW] of electricity from Belgian firms will be sold in France, while 1500 [MW] of 
electricity units will be sold in Belgium by French firms. The amounts of acid particles per 
firm in a country ([g]) , the amounts of CO particles per firm in a country ([g]), and the 
resulting amounts of smog particles per firm in a country ([g]) for game E1.8 are depicted 
in Tablej6.11 


Discussion 


The resulting electricity prices are in our case studies quite lower than the prices appearing 
in the actual European electricity market (see [32] ). The trends in the prices coincides with 
the actual situation in the European electricity market, though. The emission levels were 
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Game E1.8 acid CO smog 

BEL 3.5961-10% 1.4191-109 3.5961 - 107 
DEN 1.3304.10? 3.1320-10° 1.3304-10° 
FIN 9.7799 -10* 2.0925-10° 9.7799. 104 
FRA 0 0 0 

GER 8.2164-10* 3.3121-10° 8.2164 10% 
NLD 4.3591-10*  1.6014-106 4.3591-104 
NOR 0 0 0 

SWE 6.2383-10° 1.1744:10°  6.2383.10? 
total 2.6708-10° 9.9127-10° 2.6708- 105 


Table 6.13: Game E1.8: Emission acid particles (g), emission CO particles (g), and emis- 
sion smog particles (9), in different countries per firm 


not compared to those from actual measurements. 


6.4 Extension: Dynamic model 


In this section we propose a dynamic extension of the model introduced in Section 

Let us consider the case that the time horizon of the model is extended one time period 
ahead. In that case, firms can aim to maximize their discounted payoff by choosing the 
amount of electricity to produce with various technologies for each time period. In addition 
to the utility from sales of the electricity in the regional markets minus the average variable 
costs of production, firms also have to accommodate the fixed costs by financing the pur- 
chase of new production capacity via investments. The following equation expresses the 
payoffs in the next time period: 


5 1 
PUE Yn Y bjosga- Yn Y, (Esas) “Eva 
leL TER leL r'eR \iel iel 
pL (6.13) 
TE 


with 477° defined later by (6.15). 

The variables in (6.13) are defined as in Section with “~” identifying the variable 
for the next time period. Hence, the investment decision of the firm in the current period 
depends on the expected outcome in the next period. That is why the prices, the supply, the 
generation, and the production capacity are assigned to the next period and discounted with 
interest rate B. In addition to serving the market in the next period, firms also need to con- 
sider the value of installed capacity in the next period, while new investments are needed to 
keep the production capacity at a desired level. Parameter V; [euro/kW] represents the value 
of technology i, while diy [GW] denotes the amount of the new production capacity of 
electricity (chosen by the producer). This means that the maximum production capacity is 
no longer fixed. The firms make their investment decisions in every time period based on the 
most recent information (cf. the feedback information structure, see IS). Equation (6.14) 
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describes the capacity expansion, in which capital is depreciated at technology-specific de- 
preciation rate 9j: 


dif A) di. (6.14) 


Because of environmental considerations with respect to nuclear waste and physical limita- 
tions on the capacity of the used technologies (e.g., it is natural to restrict the capacity of 
coal power plants), the following inequality is included into the model: 


«(x dr elo) ge e 20, 9>0. (6.15) 


feF feF 


Here the shadow price $; might become nonzero, once the planned expansion of capacity 
of a certain technology reaches the maximum allowable installed capacity go [GW]. 

The producer maximizes the net profit LEN by ajoint choice of the investment decision 
and his/her production of technologies for possible regions and both load types in the next 
period. Therefore, the net profit can be defined as: 


= Tp 2" Y, Bras) - Y. Y. (Zeus 
lEL TER lEL r'eR Niel 
+) Van -L Vai LY Y fus E usi dif 
iel 1+B IEL iel HER 
1 
oR hy) Lira | Y Vapi b} } Gen je 
IEL rERrER fleF iel geF, iel 
1 = 
-5 Y Ke » hi Y L Y, GiGi." ea É 
keK leL reRicl f'eF 
= z 6: ES qi" +( (1—8 DE qr e : (6.16) 
Pa feF 


The derivative of (6.16) with respect to production leads to the following first-order condi- 
tion for the next period, which is equivalent to (6.10) in the static case: 


2 P 2 P _ Xp ul 
0 € dipsa (s Fi, fl M Wr x Kof, —(1— ày) Bra [ = uu ; 
keK ; 


0< ifr (6.17) 


The derivative of (6.16) with respect to the investment in new capacity, leads to the following 
additional first-order condition: 


Gif (Ersen aspa) >0 


leL 
gesn (6.18) 
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Equation shows that firm f makes an investment (as) as long as the marginal 
J Jiel 


return on capital Ye; hj ii; fı exceeds the marginal cost of capital DV; + (1 + B) $;. 

From an economical point of view, the value of the shadow price of the capacity usage 
iig, provides the signal to what extent a firm would like to use a particular technology 
during a particular load period. 

The amount of production capacity is no longer constant in the dynamic model and 
equation can be rewritten in terms of the decision variables as follows: 


iif. | Y dir (1-8) aff — a) 20, figs >O. (6.19) 


VER 


6.5 Conclusions and future research 


Conclusions 


We have proposed a model of the liberalized European electricity market, consisting of 
eight European countries. In the model emission limitations can be set as well as maxi- 
mal transmission capacities between the neighboring countries. The aim has been to see 
how different the electricity prices will be in a monopolistic, a duopolistic, and a perfectly 
competitive situation. 

Although the considered model is rather simple, some interesting phenomena can be 
observed: 


e The electricity prices become lower when cross-border electricity transmissions are 
allowed. 


e In the monopolistic and the duopolistic situation the electricity prices are higher than 
in the situation with perfectly competitive market. 


e Generally in the perfect competition the producers tend to use cheaper and non- 
environmentally friendly means of electricity production. The emission restrictions 
are needed to motivate the electricity producers to act more ecologically. This in- 
creases the electricity prices, though, especially in the countries with a low number 
of hydro and wind power plants. 


The extended variant of the model can be used for monitoring and predicting the behavior 
of the European electricity market. 


Model limitations and future research 


The major limitations of the model can be listed as follows: 


e Only three possible games were considered in each of the case studies: perfect com- 
petition, Stackelberg game with one leader, Stackelberg game with two leaders. Al- 
though the aim of liberalization is to obtain a highly competitive market, it will never 
be perfectly competitive. Situations with noncooperative electricity producers, in 
non-perfect competition have to be considered to obtain more realistic outcome. 


6.5 Conclusions and future research 111 


e Only eight countries were included in the model. 


e Cross-border ownerships of the electricity producers are not allowed in the model, 
while in reality they appear more and more often. 


e The electricity price is assumed to be constant within one country, while in reality 
this price might differ per the electricity producer [83]. 


e The model is very simple, while some of the data used for the modeling are real, the 
assumptions on the players' behavior are very strong. 


These limitations will be resolved in our future research. 


Practical relevance of the outcomes of the research presented in this chapter 


The liberalization of the European electricity market draws the attention of many research- 
ers. There are numerous attempts to model the current situation in order to predict the 
possible consequences of liberalization. 

In this chapter we have tried to model the electricity market in eight European countries. 
The model is quite simple and does not take into account all factors that can influence the 
liberalization process throughout Europe. Still, outcomes of our case studies coincide with 
practical observations. Although the resulting prices of our case studies are remarkably 
lower than in the current European electricity market,? their structure of prices coincides 
with other relevant studies in the same field bl [56]. The dynamic model, describing 
the current situation in a more realistic way, is being developed. 


3We assume that there are some other factors, not included into our modeling, influencing the electricity price. 


Chapter 7 


Theory of Incentives 


In this chapter we will introduce some problems from the so-called theory of incentives and 
view them as specific problems of the inverse Stackelberg type. 


7.1 Introduction 


Incentive theory emerges with the division of labor and exchange. The division of labor 
induces the need for delegation. The first contracts probably appeared in an agricultural set- 
ting, when landlords contracted their tenants. Adam Smith 721 recognized the contractual 
nature of the relationship between the masters and the workers. He recognized the conflict- 
ing interests of those two kinds of players and recognized that bargaining power was not 
evenly distributed among them; the masters generally had all the bargaining power. Smith 
also stressed the agent's participation constraint, which limits what the principal can ask 
from the agent. Although Smith did not have a vision of economic actors as long-time 
maximizers of utility, his work was important as a headstone of incentive theory, since he 
discussed the problems associated with price-rate contracts in the industry. 

Barnard igi is the one who can be credited with the first attempt to define a general 
theory of incentives in management. Even much earlier, Hume wrote the first explicit 
statement of the so-called free-rider problem. With the beginning of the theory of voting, the 
issue of strategic voting as a principal-agents problem was noticed (23). The first attempt to 
address the issue of strategic voting can be found in [12]. 

The notion of moral hazard, i.e., the ability of insured agents to affect the probabilities 
of insured events, was well-known in the insurance profession 126, 34] : 

In the regulation literature was put in the framework of the principal-agent literature 
with adverse selection by stressing the lack of information of the regulator. The problem 
was transformed into the second-best problem by weighting the firm's profit with a smaller 
weight than consumers' surplus in the social welfare function maximized by the regulated 
firm (9). In the model featuring both adverse selection and moral hazard was intro- 
duced. The ex-post observability of cost made the model technically an adverse selection 
model, though. 
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7.2 Preliminaries 


Principal-agent models fall within the economic theory of incentives or contracts 
70) , which forms a subset of the one-leader-one-follower inverse Stackelberg games intro- 
duced in Chapter[3] 

Let us consider a bilateral relationship in which a principal ? contracts an agent A to 
delegate the production of some good.! Of course, the principal has to pay the agent for 
the good. The salary which the principal offers to the agent for the production of q € R+} 
products will be t € R‚ [euro]. The variables q and t will be called quantity and transfer, 
respectively. The principal draws up a contract in which he specifies q and f. We call 
this contract the (q,t)-contract. We assume that it is always the principal who draws up the 
contract and then presents it to the agent, who, after having studied the terms of the contract, 
must decide whether or not to sign it.? We talk about a take-it-or-leave-it contract, since its 
terms are non-negotiable. 

The agent's efficiency in producing the good is determined by how much money he/she 
needs to produce one product. We will denote this value by 0, 0 € © C R+, and call it the 
agent's marginal cost. If A has the marginal cost 0 € ©, we refer to him as an agent of type 
0, or as a 0-agent. The principal does not always know the value of 0, but he does know 
the set O. The agent may pretend to be an agent of a different type. We call the agent's 
pretending to be an agent of a different type mimicking. The 0-agent announces that he is 
of the type ua € Da C Ry. We assume © = Da. Then his utility (“surplus value") from 
signing the (q,t) - contract is 


Ja (q (ua) t (uA) ,9) =t (uA) — CA (q (ua), 8) =t (ua) — 9g (ua). (7.1) 


35. € 


Here we assume that both t = t(u4) and q = q(ua) are dependent only on the agent's “an- 
nounced type". Another possibility is to consider t dependent on q (see (Sil). 

The agent will not sign the contract if 74 (g,1,0) < 0. If 4 signs the contract, he will 
produce the demanded number of goods. 

The principal’s utility function is 


Jp (q,t) = Cp(q) —t, (7.2) 


where Cp(q) describes the principal’s value of g products. This function is assumed to 
satisfy the following natural properties: 


The marginal value of the good for the principal is, thus, positive and strictly decreasing 
with respect to q on R4. 

The situation in which both ? and 4 know 0 before P offers the contract is known 
as a situation with complete information (to be studied in Section [7.3), while the situation 
in which only the agent knows his own type before the contract is designed is called a 


! We confine the agent to produce good only, although the formulation used in this chapter has also more general 
interpretation. 
?We do not permit A to make a counter-offer to ?, a situation which is known as bilateral bargaining (55), 


82]. 
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situation with adverse selection (Section[7.4). The situation in which the agent can perform 
some unobserved actions after the contract is signed is known as moral hazard (see e.g. 
iso. In this paper we focus mainly on the adverse-selection-principal-agent model. 

It is usually assumed that ? chooses among the two possible strategies: 


e The principal will offer the contract to 4, no matter how efficient 4 is (contract 
without shutdown). 


e The principal will contract 4 only if 4’s marginal cost is higher than some certain 
value (contract with shutdown), otherwise no contract will be offered. 


We will consider only contracts without shutdown. 


7.3 Complete-information principal-agent model 


Let us first assume that the agent type is from the discrete type set O E [8 0] . In this model 
the principal knows the agent's type 0 € [6.0] ,9 < 9, 0,0 € R4; hence, he can set up a 
contract slightly exceeding the agent's zero utility level and ensuring the highest possible 
utility for himself. The optimal q and t will be called the first best quantity and transfer and 
will be denoted by an asterisk. 

The principal maximizes 


subject to the agent's participation constraint 


JA (q(8),t(8),8) =1(8) — q(8)0 0. 


The principal's optimal strategy is then wp (6) “ (q* (0),:* (0)) , where (with € | 0) 


t*(0@)=q4(0)0 +£, g*(0) = en Cp(q(0)) — q (0) 9 — e, (7.3) 
q 
with ua = 0, 0 € ©. Thus, 


elc =9. (7.4) 


The agent will accept ?’s offer and gain € | O utility. 


Remark 7.1 In the following example we will assume that 4 is of type 0 from the two- 
element set © = (0,01, 0 > 0 > 0. If 


0— 8 we say, that 4 is efficient; 
~ | 9 we say, that 4 is inefficient. 


We will denote the transfer and the quantity offered to the 0-agent by t = 1(0), q = t (0), 


and to the 0-agent by 7 = (8), 7 =1(0). We will refer to the contract offered to the 0-agent 
as the (q,t)-contract and to the contract offered to the O-agent as the (7,7)-contract. 


116 7 Theory of Incentives 


Example 7.1 

The employer of a factory (P) delegates to a worker (4) to make a certain number of prod- 
ucts. The principal's objective function is given as Cp (q(0)) fin (q(80) +1). The agent can 
be only of the 0 or 0 type, where 9 = 0.1 [euro], 0 = 0.2 [euro]. The principal maximizes 
his utility function 


Je (q(8),t(8)) = Cp (q(8)) — t (8) 
= In (q(0) + 1) —t (q(0)) 


subject to t (0) — 0g (0) > 0, where 0 € {0,6}. From it follows that the principal offers 
t* (0) satisfying t* (0) = 04" (0) +£, where € | 0, and q* (0) satisfies 


x 1-0 
Er 


Hence, ? demands q*(0) = 9 products for t*(8) = (0.9 +e) [euro], if 9 = 9 = 0.1 [euro], 
and ? demands q*(0) = 4 products for r*(0) = (0.8 +£) [euro], if 0 = 0 = 0.2 [euro]. The 
agent's profit is always € | 0 and ?’s profit is In(10) — 0.9 — e = (1.4 — €) [euro] if A is 


efficient, and In(5) — 0.9 — e = (0.71 — £) [euro] if A is inefficient. 


7.4  Adverse-selection principal-agent model 


Under the adverse selection the principal is not aware of the agent's type 0 c O = [0,0] 
before writing the contract, but he/she does know ©. 


The following example shows the more specific situation with © = { 6,6}. 


Example 7.2 
Let us assume that the principal from Example [7.1] does not know the agent’s type (while 
knowing both the ® and 0 values.). He designs the pair of contracts 


(Zee), (5 1-0+e)} 


with 0 = 0.2 [euro], 0 = 0.1 [euro], hoping that each agent will pick the contract matching 
his type. If A is efficient, it pays for him to pretend to be an inefficient agent to obtain utility 


JA (0,7,7) =t@) -79 = (1-8 +e) x um (0.4 +e) [euro], 
while the 0-agent's utility without mimicking is 
AI us mo ud = 1-0_ 
JA (0.3.7) =1-9= (1 —0+e) — —=—0=€ [euro] 
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If the agent is inefficient, it does not pay to him to mimic the efficient agent, because 


JA (6,4(0),:(8)) = (8) -q0 


= (1-6 +e) - 378 =E [euro], 
Ja (8,q(8),1(8)) —1(8) — q9 
= (1-—8+¢)- 1~ 85 _ (e— 0.9) [euro]. 


When ® = [0,0], the principal has an a priori belief about the agent's type. This belief 
is embodied in the probability distribution f with cumulative distribution function F on 0, 
which will be called a principal prior. 

The principal offers the contract variables as mappings from ©. 

The principal offers the (q(-),t(-))-contract where q(-) : © — R+, t(-): 9 — R+, hoping 
that every 6—agent (6 € ©) will choose the (q (8) jt (8) )-contract. Thus, q and t become 
functions of the agent's possible type space. These functions are known before the agent an- 
nounces his type. The mechanism of announcing transfer and quantity as functions from an 
EL decision space before the contract is signed is called a direct revelation mechanism 
[51] 


Definition 7.1 

A direct revelation mechanism is a mapping Ye (-) : O — Dp, where yp = (q(-).t(-)) 
V0 € ©. The principal commits to offering the transfer (6) and the production level q(8 
the agent announces 0 € ©. 


for 
)if 


For the sake of simplicity, we assume that q(-), t(-) are differentiable with respect to each 
0 c 8. 

The direct revelation mechanism is said to be truthful, if an agent of any type from © 
does not wish to mimic an agent of a different type. 


Definition 7.2 
A revelation mechanism yp (-) is truthful if it satisfies for every 6, 6 from ©, 6 < 6 the 
following incentive compatibility constraints 


t (8) — õq (6) > t (6) — 6g (6), (7.5) 
t (ô) — ôq (ô) > z (6) — ôq (8) , (1.6) 
respectively. 
By adding and we obtain 
(6-6) (4() — q(6)) > 0 (7.7) 
for all (6,6) € © x ©. Because holds for all 6, 6 € 6, also 
da) 2; (7.8) 


dd 
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Therefore, if the direct revelation mechanism is truthful, q(-) is non-increasing. 
Inequality implies that the following first-order condition for the optimal response 
ua chosen by type 6 is satisfied: 


— (uA) — 9— (uA) =0 (7.9) 


dt a, adq a 
= 8) - 07,6) =0. (7.10) 
The local second-order condition 
dt «do x 
6) - 85.6) <0 (7.11) 


has to be satisfied as well. By differentiating (7.10), condition can be rewritten in a 
simpler way as 
BEL >0. (7.12) 
dð = 

If and are satisfied, the O-agent does not want to mimic an agent of an other 
type locally. To prevent the 0-agent from global mimicking, too, the following constraints 
must be fulfilled: 

t(0) — 0q(8) > t(uA) — 8q(uA) (7.13) 
for all (0,44) € O x ©. 

By integrating formula we obtain 


A 


t(0) — ung (ua) =t (ua) — 9q (ua) + (0— uA) q (ua) - f atoar, (7.14) 


where (0 — ua) q (ua) — dus q(t)dt > 0, because q(-) is non-increasing. 

Thus, can be extended globally. Truthful revelation mechanisms are then charac- 
terized by the two conditions and (7.12). 

We now introduce the concept of information rents. Under complete information intro- 
duced in Section[7.3]the principal is able to maintain all types of agents at their £—utility 
level, 


JA (q* (0),7*(0),0) = 1" (0) — 0q* (0) =€. 


Under incomplete information this will be not possible anymore, at least when the principal 
wants all types of agents to sign the contract offered. Let us take the revelation mechanism 
Ye (-) = (q(-).1(-)) and consider the utility that the 0-agent gains by mimicking a uA-agent, 
ua > 0 (with D = uA — 0): 


I (q(ua),t(uA),9) = t(uA) —B-qlua) =t(ua) — ua q(ua) +Dq(ua) (7.15) 
= Ja (q(ua),t(ua),ua) + Dq(ua). (7.16) 


Even if the u4-agent's utility is reduced to its lowest value &, the 0-agent benefits from an 
information rent Dq(uA) coming from his ability to possibly mimic a less efficient agent. 
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So as long as the principal insists on a positive quantity from the ®-agent, the principal 
must give up a possible rent to any other type of agent. The information rent is generated 
by the information advantage of the agent over the principal. The principal's problem is 
to determine the smartest way to give up the rent provided by any given incentive feasible 
contract. We will use the following notation: The information rents for a 0-agent will be 
denoted by Jp, i.e., Ig EIN (q(0),1(0),0) = (B) — 0q (0). 

The optimal solution of the adverse-selection-principal-agent model will be called the 
second-best solution (as opposed to the optimal solution in the situation with complete in- 
formation, which is often called the first-best solution). This second-best solution will be 
denoted by SB. From it follows that the optimal strategy for any B-agent is to play 
we = 0. The principal is aware of this. 

With the use of (7.13), the local incentive constraint can be written as 


d Ig dt dq 
NE — (0)—-0— (0) = — : 7.17 
L6 — a8). St (9) 0440) = -4(0) 15 
Thus, the principal’s problem becomes 
8 
max f (C»(a(®))-1(8)) £ (0)49, (7.18) 
[aC)40)) JO 
subject to 
d Ig 
—- = —q(0 7.19 
qg 40), (7.19) 
dq 
— « 7.2 
zo ©) € 0, (7.20) 
1g» 0 for VOEO. (7.21) 
Equation can also be rewritten as 
9 
max | (Cr (4(9)) ~ 94 (8) - 19) f (640 (7.22) 
{arj} 48 


with the use of information rent Ze. 

With the use of (7.17), the participation constraint simplifies to Ig > 0. Clearly 
the @-agent will obtain 7 (q(0),:(8),9) = rg = £. For the sake of simplicity, we will not 
consider the constraint now and check if this constraint is satisfied after finding the 
optimal strategy for the principal. 

Equation can be rewritten as follows: 


3 
Ig- Ip = — f q(t)dt, (7.23) 


Le. (with 5 = €), 


8 
Ig = f q(v)d1 4- €. (7.24) 
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The principal's objective function becomes 
8 
[ (cola) - 000) - f amar-e) Odo (7.25) 
9 F(0 
= (o (q(8)) — (e+ Eg q (6) -e) f (0)40. (7.26) 
9 f() 
Point-wise minimization of (7.26) leads to the optimal solution for the principal q?P(.) : 


S8 (gy) — 9 4 E9 
(q (©) = 0+ oe): (1.27) 


All the agents’ types obtain a positive utility by playing we =: 


d Cp 
dq 


Ja (4(8),1(8),0) = n iere (7.28) 


If the so-called monotone hazard property 


a EMI 2-0 
de \ f(0)/ ~ 
holds for all 0 € O, the solution q°?(-) satisfying (7.28) will be decreasing, and the omitted 


a is satisfied. The monotone hazard property is satisfied for most single-peak 
densities Ifi . 


7.5 Conclusions and future research 


We have proposed to view the adverse-selection-principal-agent model as a special case 
of a one-leader-one-follower static inverse Stackelberg game. Starting from the complete- 
information-principal-agent model we showed that only the least efficient type of agent 
will gain the same profit (€ [euro]) whether signing a complete-information contract or 
an adverse-selection contract. Agent of any other type will be better signing an adverse- 
selection contract. Dynamic contracts are a subject for future research. 


Chapter 8 


Conclusions and Future Research 


This chapter summarizes the research proposed and developed throughout this thesis. Its 
scope and main contributions to the current state-of-art in game theory, traffic control, elec- 
tricity market theory, and theory of incentives are briefly discussed in Section The 
future research possibilities and directions are discussed in Section[8.2] 


8.1 Contributions to the state-of-the-art 


Game theory is a widely used and investigated field. Although this field has been exten- 
sively studied and in recent years the focus has been directed more towards game theoretic 
applications than towards fundamental research, there are still game theoretic areas that have 
not been studied in a sufficient detail and, therefore, almost no theoretical results in these 
areas are known. One of such fields is the field of the so-called inverse Stackelberg games. 
In Chapter[3]of this thesis these games were defined and their properties were studied. Ap- 
plications of the Stackelberg and the inverse Stackelberg games in the static optimal toll 
design problem, the dynamic optimal toll design problem, electricity market liberalization 
problem, and the theory of incentives (contracts) were studied in Chapters [4] [5] [6] and[7] 

Our contributions with respect to the state-of-the-art in the main topics covered in this 
thesis are the following: 


e Conducted research in the field of game theory 
We recapitulated some classical results from the field of game theory. We introduced 
the so-called inverse Stackelberg games, with clear focus on one-leader-one-follower 
and one-leader-more-followers problems. We showed a way of how to find an optimal 
strategy for the leader and presented situations in which the optimal strategy of the 
leader 


exists and is unique; 


exists and is nonunique; 
— does not exist; 


— is generally unknown. 
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Both static and dynamic problems were studied. Some general statements about prop- 
erties of the inverse Stackelberg games were made. Also some unresolved issues were 
mentioned. 


Conducted research in the field of the optimal toll design 

We proposed the concept of the traffic-flow-dependent toll in the context of the opti- 
mal toll design problem and in various case studies we dealt with finding an optimal 
strategy for the road authority minimizing his/her objective function, while the drivers 
minimized their own (perceived) travel costs. Although the extensively studied first- 
best optimal toll design problem is clearly an inverse Stackelberg game!, in the field 
of the second-best optimal toll design the concept of the traffic-flow-dependent toll 
was not introduced before. The drivers in the optimal toll design problem act as one 
super-player with traffic flows on alternative routes (where tolls are imposed by the 
road authority) being his/her decision variables. We were dealing with both static and 
dynamic variants of the optimal toll design problem. 


We considered two different situations according to the information that the drivers 
have: 


— The situation with complete information in which the drivers are aware of all 
traffic conditions and minimize their actual travel costs. In equilibrium state the 
deterministic user equilibrium (DUE) applies. 


— The situation with incomplete information, in which the drivers are not aware of 
all traffic conditions and minimize their actual perceived travel costs. In equi- 
librium state some stochastic user equilibrium (SUE) applies. As an example of 
such an equilibrium the logit-based stochastic equilibrium (LB-SUE) was used. 


The deterministic user equilibrium is a limiting case of the logit-based stochastic equi- 
librium when the so-called perception error tends to infinity. With the deterministic 
equilibrium the optimal toll design problem is analytically solvable, unlike in the case 
with the more general LB-SUE. 


Also, since DUE is a limiting case of LB-SUE, the algorithm that we have proposed 
for solving the optimal toll design problem with the second-best traffic-flow depen- 
dent toll and the drivers driven by LB-SUE can be used for solving the optimal toll 
design problem with drivers driven by DUE, too. This algorithm uses neural networks 
simulation and belongs to advanced heuristic methods, which can be efficiently used 
for solving NP-hard problems. The optimal toll design problem belongs to the class 
of this type of problems. 


We have shown that the use of the traffic-flow dependent toll may improve the system 
performance remarkably, while the traffic flow-dependent toll can never yield a worse 
outcome than the traffic-flow invariant toll. The choice of tolled links influence the 
outcome of the game remarkably. 

Theorems about the existence of the solution for the general variant of the optimal 


toll design problem have been stated for both the static and the dynamic situations. 
Case studies of various network types were presented, too. 


'So far the first-best optimal toll design problem has not been recognized as an inverse Stackelberg game, 
although it is a clear example of the game of this type. 
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e Conducted research in energy market liberalization problem 
We have proposed a model of the European electricity market. Eight countries were 
included in the model: Belgium, Denmark, Finland, France, Germany, The Nether- 
lands, Norway, and Sweden. The model uses real data about electricity production 
prices, emission factors, and electricity consumption in individual countries. Differ- 
ent types of games, differing in the following criteria, have been considered: 


form of the leadership (no leader, one leader - Stackelberg game with one leader, 
two leaders - Stackelberg game with two leaders); 


type of the competition among the leaders and the followers (perfect vs. imper- 
fect); 


role of borders (game with the cross-border electricity transmissions allowed vs. 
game with no cross-border electricity transmissions); 


— role of emissions (emission constraints included or no emission constraints). 
As a result of the case studies we have drawn the following conclusions: 


— The electricity prices are the highest if one of the electricity producers acts as 
a leader, i.e., has a monopoly in his country. The prices decrease with two 
competing leaders and are the lowest when none of the electricity producers 
acts as the leader and perfect competition takes place. 


— While perfectly competitive electricity market increases the emission factors 
when emission restrictions are not imposed, a right choice of emission con- 
straints may decrease emission factors, while the electricity prices do not in- 
crease that much. 


— The electricity prices decrease if the cross-border electricity transactions are 
allowed. 


The outcomes of our model coincide with the experiences in the real electricity mar- 
ket. 


e Conducted research in theory of incentives 

The principal-agent problem from the economical theory of incentives has been iden- 
tified as an example of the inverse Stackelberg games. Various problems of this type 
have been solved, with a focus on the optimal strategy for the principal as the leader 
and interesting phenomena. The only situation, in which the principal receives posi- 
tive outcome no matter how efficient the agent that the principal is contracting is, is 
the situation with full information. While the principal-agent theory is a classical one, 
we have presented it as a special case of an inverse Stackelberg game. 


8.2 Future research 


In this section we will discuss possible future research directions for each of the main topics 
addressed in this thesis. 
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e Future research in Game theory 
While in Chapter [3]important phenomena in inverse Stackelberg games were intro- 
duced, mainly by means of specific examples, some general properties have not been 
studied yet. The following topics are interesting subjects for future research: 


Existence of the solutions to general problem of the inverse Stackelberg type. 
An inverse Stackelberg game with a higher number of players. 


Inverse Stackelberg game with leaders or followers being cooperative among 
themselves. 


The problems with incomplete information. 


e Future research in Optimal toll design problem 
The problems, that should be addressed in the future research, are: 


The optimal toll design problem with traffic-flow-dependent toll with elastic 
demand. 


The problems with heterogeneous user classes of the drivers. 


The problems with the traffic-flow dependent tolls that are not polynomial func- 
tions of the traffic flows. 


The optimal toll design problem with the drivers driven by different user equilib- 
rium than LB-SUE. There exist more realistic models of the travelers' behavior, 
where the travelers are driven by the equilibria that are extensions of the LB- 
SUE. Also probit-based models can be used. The problem becomes difficult to 
solve in this case, though. We expect that also in this situation the problem the 
traffic-flow dependent toll brings better outcomes for the road authority. 


The dynamic optimal toll design problem with the travelers’ departure time 
choice. 


e Future research in Energy market liberalization problem 
The issues that deserve future research are: 


Dynamic model - although a possible extension of the current model to the situ- 
ation with more time steps (discrete dynamic model) was discussed, case studies 
performed were more of academic nature, while computations with “real-size” 
models were not performed. 


Incorporating more countries into the model. To be able to see the influence of 
the liberalization process throughout the Europe, all European countries have to 
be included. 


Model with elastic electricity demand. Although this option was briefly studied 
when the extension of the model to the dynamic problem was discussed. 


Game with the electricity consumers being incorporated into the model. In this 
case the electricity producers can be leaders in a Stackelberg or in an inverse 
Stackelberg game and the electricity consumers can be the followers. 
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— Including net region-specific electricity losses per country and also including 
net losses of both the countries when electricity cross-border transmission takes 
place. 


e Future research in Theory of incentives 
Important subjects for future research are: 


- Dynamic contracts. In Chapter [7]the clear emphasis was on static contracts, 
while the extension to the dynamic version of the problem was mentioned quite 
briefly. 


— Problems with moral hazard, problems combining moral hazard with adverse 
selection. 


— Problems with more principals and/or agents. 
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Samenvatting 


Over Stackelberg- en inverse Stackelbergspellen en hun toepass- 
ing in het ontwerpen van optimale tollen, de liberalisering 
van energiemarkten en in de theorie van aansporingen 


Inverse Stackelbergspellen zijn het onderwerp geworden van recent onderzoek in speltheo- 
rie. Tot nu toe stond de theorie voor dit soort spellen slechts in de kinderschoenen en was 
er dus zeer weinig bekend over inverse Stackelbergspellen. In dit proefschrift wordt inge- 
gaan op het theoretisch oplossen van zulke problemen en wordt een aantal zeer uitdagende 
problemen uit een variéteit aan domeinen in het raamwerk van inverse Stackelbergspellen 
geplaatst en opgeslost. 

In Stackelbergspellen bepaalt een zogenaamde leider acties voor één of meer zoge- 
naamde volgers. In het algemeen is het vinden van een optimale strategie voor een leider 
in deze spellen extreem moeilijk; in veel gevallig zelfs onmogelijk. Beginnend met een- 
voudige statische problemen en daarna verdergaand met meer moeilijke dynamische prob- 
lemen, wordt in dit proefschrift aangetoond hoe de optimale strategie voor een leider op een 
heuristische manier gevonden kan worden. 

In dit proefschrift wordt de toepassing van speltheorie in de volgende drie specifieke ge- 
bieden voorgesteld: bet bepalen van optimale tollen, liberalisering van de elektriciteitssector 
en de theorie van aansporingen. 

Het ontwerpen van een optimale tol wordt in de proefschrift beschreven als een spel 
van het Stackelberg type. Een wegbeheerder representeert hierbij de leider en de wegge- 
bruikers representeren de volgers. De wegbeheerder bepaalt de tol voor een aantal van de 
wegen in een wegennetwerk. De wegbeheerder doet dit op een zodanige manier dat zijn 
doelfunctie wordt gemaximaliseerd, terwijl de weggebruikers hun beslissingen maken op 
een zodanige manier dat hun reiskosten worden geminimaliseerd. Als de tol die de wegbe- 
heerder bepaalt niet afhankelijk is van de verkeersstroom, dan is het op te lossen probleem 
een klassiek Stackelbergspel. Als de tol wel afhankelijk is van de verkeersstroom, dan is het 
probleem een invers Stackelbergspel. In dit proefschrift wordt een optimale stroomafhanke- 
lijke tol voor de wegbeheerder gevonden, voor zowel statische als dynamische varianten van 
het tolontwerpprobleem. Als het oplossingsconcept voor de weggebruikers wordt bepaald 
met behulp van een zogenaamd deterministisch gebruikersequilibrium, dan kan het prob- 
leem analytisch worden aangepakt. Als het zogenaamde stochastische gebruikersequilib- 
rium wordt gebruikt, dan moeten numerieke methoden worden gebruikt om een oplossing 
te vinden. Aangezien dit probleem NP-moeilijk is, stellen wij voor om een oplossingsaan- 
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pak te gebruiken die gebaseerd is op neurale netwerken. We vergelijken de uitkomsten van 
de spellen met stroomonafhankelijke tol en stroomafhankelijke tol. We concluderen dat de 
stroomafhankelijke tol de prestaties van het systeem significant kan verbeteren. Daarnaast 
worden interessante eigenschappen van dit probleem beschreven en bediscussieerd. 

De liberalisering van de elektriciteitsmarkten is in dit proefschrift gedefinieerd als een 
competatief spel tussen elektriciteitsproducenten in 8 Europese landen. Elektriciteitsvraag 
wordt hierbij als gegeven beschouwd. De producenten maken een keuze uit de inzet van 
beschikbare middelen voor elektriciteitsproductie en de hoeveelheid te produceren elek- 
triciteit op een zodanige manier dat hun winst wordt gemaximaliseerd. Verschillende spelsce- 
nario's worden beschouwd: Perfecte competitie, een spel met één leidende producent per 
land en een spel met twee leidende producenten per land (waarbij de leiders onderling een 
zogenaamde Nashstrategie gebruiken). De uitwisseling van elektriciteit tussen naburige lan- 
den is toegestaan en beperkingen op emissies worden meegenomen. Een numeriek model, 
gebruikmakend van realistische data, wordt voorgesteld om het probleem op te lossen. Onze 
resultaten suggeren dat de liberalisatie van de elektriciteitsmarkten tot een daling in de prijs 
voor elektriciteit kan leiden. 

Ten slotte behandelen wij zogenaamde principal-agent modellen uit de theorie van aans- 
poringen als een speciale groep van inverse Stackelbergspellen. In dit geval is de principal 
de leider an de agent de volger. De leider contracteert de volger met het doel om een bepaald 
aantal goederen te produceren. De mate van effectiviteit van de volger kan variéren. Deze 
effectiviteit is onbekend bij de leider. Het probleem van het vinden van een optimale strate- 
gie voor de leider wordt behandeld. Interessante fenomenen in dit spel worden gepresen- 
teerd en een optimale strategie voor de leider wordt afgeleid. 


Katerina Staňková 


Summary 


On Stackelberg and Inverse Stackelberg Games and their 
Applications in the Optimal Toll Design Problem, the En- 
ergy Markets Liberalization Problem, and in the Theory of 
Incentives 


Inverse (or reverse) Stackelberg games have become the subject of recent game theory re- 
search, as a special type or as an extension of Stackelberg games. So far, only very little 
theory about inverse Stackelberg games is available and the available theory is still in its 
infancy. In this thesis we focus on theoretically solving such problems and we propose to 
treat several challenging problems in various fields inside this framework. 

In Stackelberg games a so-called leader determines actions for one or more so-called 
followers. The problem of finding an optimal strategy for the leader in these games is 
in general extremely hard to solve, and often even completely unsolvable. Starting from 
simple static problems and proceeding to more difficult dynamic ones, we show how to find 
the optimal strategy for the leader in a heuristic manner. 

In this thesis, the application of game theory is proposed in the following domains: The 
optimal toll design problem, the electricity markets liberalization problem, and the theory 
of incentives. 

The optimal toll design problem is a game of the Stackelberg type in which a road au- 
thority acts as the leader and drivers in the road network act as the followers. The road 
authority sets tolls on some of the links in the network in order to maximize its objective 
function, while the drivers make their travel decisions in order to minimize their perceived 
travel costs. If the toll that the road authority sets is traffic-flow invariant, the problem is 
the "classical" Stackelberg game; if the toll is traffic-flow dependent, the problem is of the 
inverse Stackelberg type. We determine the optimal traffic-flow dependent toll for the road 
authority for both static and dynamic variants of the problem. If the solution concept for the 
drivers’ behavior is the deterministic user equilibrium, the problem can be dealt with ana- 
lytically. If the stochastic user equilibrium applies, numerical methods have to be applied 
to find a solution. As the problem is NP-hard, we use a neural-networks based solution 
approach to solve the problem. We compare outcomes of the games with traffic-flow in- 
variant and traffic-flow dependent toll and conclude that the traffic-flow dependent toll can 
improve the system performance remarkably. Interesting phenomena in this problem and 
its properties are discussed, too. 
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The electricity markets liberalization problem is defined in this thesis as a noncoopera- 
tive game among electricity producers in eight European countries, in which the electricity 
demand is exogenous. The producers choose among available means of electricity produc- 
tions and quantities to produce in order to maximize their profit. Different game scenarios 
are considered: Perfect competition, a game with one leading producer per each country, 
and a game with two leading producers, playing Nash among themselves, for each country. 
The transmission of electricity between neighboring countries is allowed and emission con- 
straints are considered. A numerical model, using real data, is developed in order to solve 
the problem. Our results suggest that liberalization of electricity markets leads to electricity 
price decrease. 

Finally, we deal with so-called principal-agent models from the theory of incentives as 
a specific group of inverse Stackelberg problems. Here the principal as a leader contracts 
an agent as a follower in order to produce certain goods. The agent can be of different 
efficiency, often unknown to the principal. The problem of finding the optimal strategy for 
the principal is dealt with. Interesting phenomena in this game are presented and an optimal 
strategy for the leader is derived. 
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